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ABSTRACT 

A  methodology  for  improving  a  large  scale  C3  system  in  an  optimal  manner  is 
developed.  The  first  step  of  this  methodology  consists  of  a  procedure  for  designing 
experiments  to  run  on  a  large  scale  C3  system:  it  aims  at  determining  the  smallest  number 
of  experiments  that  enables  one  to  evaluate  the  effectiveness  of  an  actual  system.  The 
second  step  consists  of  a  dynamic  programming  algorithm  aimed  at  determining  an  optimal 
sequence  of  improvements  for  a  system  on  the  basis  of  the  effectiveness  analysis. 

This  methodology  is  applied  to  an  actual  air  defense  system:  each  configuration  of 
the  system  is  evaluated  with  the  first  step  of  the  methodology;  then,  the  second  step  is  used 
to  find  an  optimal  sequence  of  modifications  for  the  system  as  it  goes  from  one 
configuration  to  another.  Z'  v 
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CHAPTER  1 
INTRODUCTION 


1.1  INTRODUCTION 

A  major  focus  of  the  research  in  Command  Control  and  Communication  (C^)  is 
the  need  to  assess  quantitatively  the  utility  of  C^  systems.  Over  the  past  decade, 
methodologies  have  been  proposed  that  provide  system  developers  with  powerful  tools  for 
evaluating  the  effectiveness  of  systems  [Mishan  1976,  Dersin  and  Levis  1981,  White 
1985].  All  these  methodologies  assume  that  experimental  data  can  be  gathered  for  the 
system  to  be  evaluated. 

A  new  problem  arises  when  very  large  scale  economic,  social  or  military  systems 
are  considered  because  it  is  often  not  possible  to  run  a  sequence  of  experiments  and  collect 
the  data  necessary  to  carry  out  the  assessment.  A  second  feature  of  these  systems  is  that 
their  development  lasts,  generally,  many  years.  Indeed,  because  of  budget  and  time 
constraints,  they  are  built  progressively  as  components  get  added  gradualy.  These  large 
scale  systems  change  with  time,  and  developers  seek  to  optimize  their  evolution. 

A  common  approach  to  analyse  and  evaluate  system  design  is  to  build  a  test  bed 
which  provides  the  developer  with  the  ability  to  consider  many  different  configurations  of 
the  same  system  and  with  a  means  to  gather  data  on  these  configurations  in  order  to 
evaluate  theim.  Because  of  the  high  degree  of  complexity  of  the  system,  and  because  of  the 
need  for  fidelity  between  actual  system  behavior  and  test  bed  results,  the  test  bed  is  often 
very  complex  too.  Indeed,  it  is  a  surrogate  which  is  supposed  to  capture  all  the  properties 
of  interest  of  the  real  system.  It  contains  a  mix  of  actual  equipment  and  software  and 
simulations.  The  construction  of  a  test  bed  addresses  the  two  problems  outlined  above: 
first,  it  allows  one  to  exercise  the  system  (the  test  bed  in  fact)  in  a  reasoned  and  directed 
manner,  second,  it  enables  one  to  simulate  the  evolution  of  the  system  by  evaluating  the 
effectiveness  of  different  configurations. 

Once  a  test  bed  has  been  built,  the  challenge  is  to  determine  whether  the  actual 
system  is  effective  and  whether  and  how  it  can  be  improved.  The  developer  is  interested  in 
evaluating  the  effectiveness  of  many  different  configurations  of  the  system  and  in  selecting 
an  optimal  sequence  of  improvements  that  will  transform  gradually  the  initial  system  into  a 


more  effective  one.  For  each  configuration  of  the  system,  one  must  design  experiments  to 
run  on  the  test  bed  in  order  to  obtain  data  and  assess  the  effectivenness  of  this 
configuration.  Then,  one  must  evaluate  changes  and  determine  the  optimal  sequence  of 
configurations  (improvements)  for  the  system. 

In  response  to  these  needs,  a  methodology  is  developed  in  this  thesis:  the  first  step 
is  the  introduction  of  a  procedure  for  designing  experiments  to  run  on  a  test  bed  and 
evaluating  the  effectivenness  of  any  configuration  of  the  system;  the  second  step  is  the 
development  of  an  algorithm  for  optimizing  the  evolution  of  the  system  on  the  basis  of  the 
effectiveness  evaluation. 

This  methodology  draws  on  the  framework  first  developed  by  Dersin  and  Levis 
(1981),  and  then  applied  to  systems  by  Bouthonnier  (1984),  Cothier  (1984)  and  Karam 
(1985).  The  method  of  analysis  used  is  based  on  relating  the  performance  of  a  system  to 
the  requirements  of  the  mission  is  has  to  fulfill. 

1.2  OUTLINE  OF  THE  THESIS 

This  thesis  is  organized  into  eight  chapters  as  follows.  Chapter  2  is  devoted  to  the 
detailed  presentation  of  the  System  Effectiveness  Analysis  methodology.  In  this  chapter 
revised  definitions  are  introduced  to  enable  one  to  draw  unambiguously  the  physical  and 
temporal  boundaries  of  a  system. 

A  procedure  for  designing  experiments  to  be  run  on  a  test  bed,  and  for  evaluating 
the  system's  effectiveness  is  presented  in  Chapter  3. 

In  Chapter  4,  a  dynamic  programming  algorithm  is  given;  this  algorithm  enables 
one  to  determine  an  optimal  sequence  of  improvements  for  a  system. 

Chapter  5  is  devoted  to  the  presentation  of  an  air  defense  system  and  of  its  test  bed 
while  in  Chapter  6  the  System  Effectiveness  Analysis  (SEA)  is  carried  out  using  a 
mathematical  model  of  the  system. 

In  Chapter  7,  the  methodology  introduced  in  Chapters  3  and  4  is  applied  to  the  air 
defense  system  presented  in  Chapter  5. 

Finally,  the  basic  ideas  developed  in  this  thesis  are  summarized  in  Chapter  8; 
recommendations  for  future  research  are  also  given. 


CHAPTER  2 

SYSTEM  EFFECTIVENESS  ANALYSIS  (SEA) 


2.1  PURPOSE  OF  SYSTEM  EFFECTIVENESS  ANALYSIS 

The  objective  of  System  Effectiveness  Analysis  is  not  to  evaluate  the  performances 
of  a  system  per  se,  but  rather,  to  measure  the  extent  to  which  a  system,  given  its 
performance,  is  effective  in  meeting  the  mission  it  is  designed  to  accomplish.  One  must 
first  define  the  four  basic  concepts  introduced  by  the  theory:  the  System,  the  Mission,  the 
Environment,  and  die  Context. 

2.2  DEFINITIONS 

Before  defining  and  measuring  the  effectiveness  of  a  given  system,  one  must  first 
define  what  is  meant  by  "System".  The  goal  of  the  following  section  is  to  define  what  the 
system  is  for  which  effectiveness  is  to  be  measured. 

2.2.1  System,  Environment,  Context 

Let  us  consider  a  set  U  which  represents  the  universe:  U  will  be  called  the 
universal  set  It  may  contain  a  great  diversity  of  elements  such  as  physical  entities,  data 
bases,  or  doctrines.  A  goal  is  defined  to  be  a  particular  desired  state  of  the  universe  U. 
Typical  goals  are  :  to  produce  data,  to  transmit  information,  or  to  defend  one's  assets. 

A  system  S  is  defined  to  be  a  set  of  elements  of  the  universe  U  that  act  together  by 
exchanching  information  (connectivity)  toward  the  achievement  of  a  particular  goal.  The 
set  S  is  a  subset  of  U. 

An  element  u  of  the  universe  is  included  in  the  environment  E  if  and  only  if,  u 
docs  not  belong  to  S,  and,  the  system  can  act  upon  u,  and,  u  can  act  upon  the  system. 

The  context  C  then  is  defined  as  the  complement  of  the  set  S  »  E  in  the  universe  . 
With  these  three  definitions  one  can  easily  deduce  the  following  properties: 


S  n  C  =  0 


EnC  =  0 


(2.1) 


where  "u"  denotes  the  union  of  two  sets,  and  ”n"  their  intersection. 

One  can  represent  these  three  concepts  in  a  single  graph  as  shown  in  Fig.2.1. 


Fig.2.1:  System,  Environment,  and  Context 


Qwteijngs 

To  draw  the  boundary  of  S,  one  must  follow  three  major  steps: 

(1)  Identify  the  goal  the  system  is  designed  to  achieve 

(2)  Define  the  first  element  of  the  system;  this  element  is  closely  related  to  the 
goal:  indeed,  a  goal  being  a  particular  state  of  the  environment,  one  can  figure  out  which 
elements  of  U  act  directly  on  the  environment  to  achieve  the  goal.  One  can  denote  this 
stage  of  the  system  definition,  stage  Sq  . 

(3)  Define  in  an  incremental  approach  the  next  elements  of  S  on  the  basis  of  two 
criteria:  connectivity  with  the  components  already  included  in  S,  and  degree  to  which  a 
given  element  acts  toward  the  achievement  of  the  goal.  Let  us  suppose  that  the  system 
definition  is  at  stage  Sn_i,  then  one  must  define  stage  Sn.  To  do  so,  consider  all  the 
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components  connected  to  Sn_  j  (connectivity  criteria).  For  each  of  these  components, 
determine  whether  it  acts  toward  the  same  goal  as  Sn.j  does;  if  it  does,  this  component  is 
to  be  included  in  Sn.  If  there  are  no  more  elements  connected  to  Sn_i>  or  if  each  element 
connected  to  Sn.}  does  not  act  toward  the  same  goal  as  Sn_}  does,thcn  S=Sn_j  . 

Once  the  system  S  has  been  defined,  one  must  determine  the  environment  E.  In 
the  complement  of  the  set  S,  the  environment  is  composed  of  the  elements  of  the  universe 
that  the  system  can  affect  and  that  can  affect  the  system:  this  definition  is  simple  to  apply 
once  the  system  S  has  been  determined. 

2.2.2  Mission 


The  term  "mission"  designates  the  task  the  system  has  to  perform.  It  is  the 
particular  state  of  the  environment  that  has  to  be  achieved  by  the  system. 

The  mission  depends  on  the  system  considered:  for  instance,  the  mission  of  a  Cp 
system  is  to  provide  "adequate”  information  to  effectors  on  the  basis  of  the  data  it  receives 
from  the  sensors;  for  a  system,  sensors  and  effectors  are  part  of  the  environment, 
according  to  the  definitions  sketched  out  above.  The  mission  of  the  forces  this  system 
supports  is  defined  in  term  of  the  environment  of  the  forces  which  is  different  from  that  of 
the  C?  system  itself. 


2.3  TEMPORAL  BOUNDARY  OF  A  SYSTEM 


2.3.1  Introduction 


In  the  previous  section.  System,  Environment  and  Context  have  been  defined  in 
order  to  apply  the  System  Effectiveness  Analysis  methodology  to  well  defined  situations; 
however,  these  definitions  deal  only  with  static,  or  non-evolving  systems.  In  this  section, 
these  definitions  are  extended  to  account  for  the  particuliar  features  of  systems  that  change 
as  time  goes  by  (evolving  systems). . 

These  definitions  will  enable  one  to  draw  the  temporal  boundary  of  a  system. 

2.3.2  System,  Environment  and  Context  as  fuzzy  sets 

In  section  2.2,  System,  Environment  and  Context  have  been  defined  as  subsets  of 
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the  universal  set  U.  One  can  extend  these  definitions  to  fuzzy  set  ones  [Dockery  1985]. 

Two  membership  functions  ,  pg  and  pj^  can  be  defined  for  each  element  of  the 
universe:  they  represent  the  fuzzy  set  extensions  of,  respectively,  the  system  and  the 
environment 


0<p§<l  ,  0<pg<l 


(2.2) 


If  pg(u)  =1 ,  u  is  said  to  be  an  element  of  the  system. 

If  pg(u)  =0 ,  u  is  not  element  of  the  system. 

Otherwise,  u  belongs  only  in  part  to  the  system;  this  may  be  the  case  for  a  data-base,  a 
computer,  or  a  communication  link  shared  by  two  distinct  systems. 

Similarly,  an  element  u  of  the  universe  may  be  partially  included  in  the 
environment 

Since  an  element  that  is  included  neither  in  the  system  nor  in  the  environment 
must  be  included  in  the  context,  the  membership  function  of  C  is  defined  by: 

tiC  =  1  -  M-S  *  tiE  (2.3) 

E*aropte 

For  an  anti-aircraft  battery  included  in  a  larger  system  S,  the  membership  function 
decreases  as  missiles  get  fired,  and  not  replaced,  because  the  contribution  of  the  battery 
(connected  to  the  large  system)  to  the  overall  goal  of  S  goes  down. 

Fuzzy  sets  will  be  used  in  defining  evolving  systems. 

2.3.3  Evolving  Systems 

A  system  is  defined  by  its  membership  function  pg  which  may  vary  over  time.  If  i 
is  an  index  of  the  elements  in  the  universe  U,  let  us  consider  two  families  of  real  positive 
numbers  that  are  inferior  to  1  :  (gsj)i  and  (/sj)j . 

A  system  S  is  said  to  be  "non  evolving"  during  the  time  interval  [t',t"]  if  and  only 
if,  for  all  elements  uj  in  the  universe  and  for  all  time  t  included  in  [t',t"] : 

gsi>  H(uj)>  /si  (2.4) 

The  bounds  gsj  and  /sj  are,  respectively,  the  upper  and  the  lower  bound  of  the 
membership  functions  for  each  element  (indexed  by  i)  in  the  system. 
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For  example,  a  computer  with  five  processors  fnay  be  included  in  a  particular 
system,  if  at  least  three  of  its  processors  are  working;  otherwise,  the  computer  is  not 
included  in  the  system  because  its  contribution  to  the  overall  mission  is  assumed  to  be 
negligible  if  two  or  more  processors  are  out  of  order.  This  fact  can  be  modeled  by  a 
membership  function  and  a  range  of  variation  for  this  function. 

If  the  universe  has  only  two  elements  uj  and  U2,  one  can  define  a  non-evolving 
system  by  a  membership  function  and  a  set  of  bounds  (gi,g2^i^2^  included  in  [0,1]^. 

gsi>  M-(Ui)  >  /si  i-l>2  (2.5) 

The  definition  of  a  non-evolving  system  can  be  further  extended  to  a  "quasi-static 
non-evolving  system"  (Fig.2.2),  by  allowing  the  two  families  (gsj)j  and  (/s j)j  to  vary  over 
time;  i.e.  the  upper  and  lower  bounds  for  the  membership  functions  are  allowed  to  vary. 


M(u) 


Fig.2.2:  Quasi  static  non-evolving  system 


In  Fig.2.3,  the  system  evolves  at  time  t. 


Fig.2.3:  Evolving  system 


2.3.4  Conclusion 

The  definitions  provided  in  this  section  are  useful  for  drawing  the  temporal 
boundary  of  a  given  system:  Indeed,  the  core  of  the  SEA  methodology  deals  with  static 
systems  whose  temporal  boudary  must  be  determined  before  going  into  the  details  of  the 
analysis.  The  above  definitions  enable  one  to  determine  this  boundary  in  a  systematic 
manner. 


2.4  PARAMETERS  AND  MEASURES  OF  PERFORMANCE 

Parameters  are  the  independent  variables  in  system  effectiveness  analysis.  The 
system  parameters  are  entities  whose  value  determine  system  behavior  and  specify  system 
structure.  They  are  defined  within  the  system  boundary.  Environment  or  context 
parameters  refer  to  the  independent  variables  that  describe  the  environment  or  the  context. 

Measures  of  performance  (MOPs)  are  measurable  quantities  that  describe  system 
properties  or  attributes.  Their  values  depend  on  the  values  of  the  parameters  that 
characterize  the  system,  the  environment  and  the  context. 


2.5  SYSTEM  LOCUS 


The  system  measures  of  performance  vary  in  Q,  a  subset  of  Rn  where  n  is  the 
number  of  MOPs.  Since  the  parameters  are  constrained  to  be  in  a  subset  P  of  RP  where  p 
is  the  number  of  parameters,  one  cannot  expect  the  MOPs  to  take  any  value  in  O.  Since 
each  MOP  is  a  function  of  several  parameters,  one  can  define  a  mapping  from  the 
parameter  space  into  the  MOP  space  (Fig.2.4):  This  mapping  is  obtained  by  exercising  the 
system  (or  by  running  simulations)  for  different  contexts  and  different  values  of  the  system 
parameters  (Fig.2.4)  in  order  to  generate  the  reachable  MOP  values.  The  set  of  this 
reachable  MOP  values  is  the  system  locus  Ls. 


Models 

System 

or 

MOP’S 

Parameters 

Data 

* 


Context  • 


Fig.2.4:  Generation  of  MOP  values 

This  mapping  defines  the  system  locus  Ls.  It  is  the  subset  of  Q  such  that  for  each 
X  in  Ls  there  exists  p  in  the  parameter  space  such  that  x=x(p)  (Fig.2.5). 

2.6  MISSION  LOCUS 

The  mission  the  system  has  to  achieve  is  defined  by  requirements  in  the  MOP 
space.  These  requirements  are  obtained  by  running  models,  games  or  plans  for  different 
contexts  and  for  different  mission  parameters  (Fig.2.6). 

In  order  to  enable  one  to  compare  the  mission  and  the  system  in  the  next 
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subsection,  the  requirements  obtained  by  running  models  in  Fig.2.6  must  be  expressed  in 
term  of  the  MOPs  defined  for  the  system:  the  mission  MOPs  (expressing  the  mission 
requirements)  must  be  the  same  as  the  system  MOPs  (expressing  the  system  capabilities). 

The  set  of  MOP  values  that  satisfy  the  mission  requirements  constitutes  the 
mission  locus  (Fig.2.7).  The  mission  locus  can  be  written  as  follows: 

a^<  gj(i)  <ty  ,  i=l,2,...,n  (2.7) 


Fig.2.7:  Mission  locus 


2.7  MEASURES  OF  EFFECTIVENESS  (MOEs) 

Measures  of  effectiveness  are  derived  from  the  comparison  of  the  two  loci  Ls  and 
Ljjj  .  Qualitatively,  the  greater  the  intersection  of  the  two  loci  ,  the  more  effective  the 
system  is.  If  V(L)  is  a  measure  on  the  locus  L  (Fig.2.8),  one  can  define  the  following 
MOEs: 


Ej  =  V(LS  n  L^/V (Ls)  (2.8) 

E2  =  V(LsnLm)/V(Lm)  (2.9) 

E3  =  V(LS  n  LmmLs  u  1^)  (2. 10) 

E^  is  the  degree  to  which  the  system  capabilities  are  included  in  the  mission  locus; 
It  measures  how  well  the  system  capabilities  are  used  for  the  mission  considered. 

E2  is  the  degree  to  which  the  mission  locus  is  covered  by  the  system;  It  is  the 
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degree  of  coverage  of  the  mission  by  the  system. 

E3  is  the  degree  of  overlapping  between  the  system  capabilities  and  the  mission 
requirements;  I-E3  is  the  degree  of  misfit  between  the  two  loci. 

If  V(LS  r>  Ljjj)  is  not  equal  to  zero,  then  one  can  deduce  the  following  relation 
between  the  three  MOEs  defined  above: 


1/Ej  +  I/E2  =  1  +  I/E3 


(2.11) 


The  important  fact  in  the  passage  from  an  MOP  to  an  MOE  is  the  addition  of 
requirements:  Before  setting  requirements,  the  system  locus  doesn’t  tell  how  effective  the 
system  is. 


L3  r>  L 


Fig.2.8:  Measures  of  effectiveness 


Special  cases:  If  Ls  is  included  in  then  Ej=l  and  E2=E3<1.  If  is  included 
in  Ls  then  E2=l  and  Ej=E3  <  1. 

For  a  given  system  one  can  define  many  different  MOEs  representing 
effectivenesses  from  various  standpoints:  These  MOEs  are  called  partial  MOEs.  Let 
Ej,E2,...,Ep  be  the  partial  MOEs  for  a  given  system.  Debreu  [1968]  has  shown  that, 
under  certain  conditions,  there  exists  a  real  valued  function,  a  utility  function,  which  is 
continuously  dependent  on  the  Ej.  Let  U  be  such  function  taking  values  between  0  and  1 : 


[0,1  ]P  - 

Ej,E2,...,Ep 


->  [0,1] 

->  U(  E1,E2,...,Ep) 


Then,  the  global  measure  of  effectiveness  can  be  taken  to  be  U(  Ej,E2,...,Hp). 

2.8  STEPS  OF  THE  METHODOLOGY 

The  steps  of  the  System  Effectiveness  Analysis  and  their  interrelationships  are 
shown  in  Figure  2.9  and  are  summarized  below  (Step  1  to  8  are  executed  for  each 
subsystem). 

Step  1:  Define  the  System,  the  Environmnent  the  Context  and  the 

Mission. 

Step  2:  Define  from  the  system  characteristics,  the  Environment,  and 
the  Context,  the  independent  variables  (parameters)  that  arc  1 
ikely  to  influence  the  System  MOPs. 

Step  3:  Relate  the  System  MOPs  to  the  parameters. 

Step  4:  Define  from  the  mission  characteristics,  the  Environment, 
and  the  Context,  the  independent  variables  (parameters)  that 
are  likely  to  influence  the  Mission  MOPs.  In  order  to  enable 
one  to  compare  the  mission  and  the  system,  the  mission 
MOPs  must  be  the  same  as  the  system  MOPs. 

Step  5:  Relate  the  Mission  MOPs  to  the  parameters. 

Step  6:  Compute  the  System  locus. 

Step  7:  Compute  the  Mission  locus  (or  mission  requirements). 

Step  8:  Compute  the  partial  measure  of  effectiveness. 

Step  9:  Combine  the  partial  measures  of  effectiveness  obtained  for 
each  subsystem  into  a  global  measure  of  effectiveness  for  the 
system  as  a  whole. 
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Fig.2.9:  System  Effectiveness  Analysis  Methodology 


Mission 


2.9  CONCLUSION 


The  framework  of  the  System  Effectiveness  Analysis  has  been  presented 
throughout  chapter  2;  this  methodology  constitutes  the  basis  of  the  analysis  to  be  carried 
out  in  this  thesis  in  order  to  design  experiments  to  run  on  large  scale  systems  and  to 
improve  these  systems. 

Chapter  3  will  present  a  experiment  design  algorithm  to  evaluate  the  effectiveness 
of  real  systems  as  opposed  to  any  model  of  these  systems. 


CHAPTER  3 

EXPERIMENT  DESIGN  FOR  LARGE  SCALE  SYSTEMS 


3.1  INTRODUCTION 

In  this  chapter,  a  procedure  for  designing  experiments  to  run  on  a  large  scale  system 
is  developed.  This  procedure  will  enable  one  to  assess  the  effectiveness  of  an  actual  system 
and,  eventually,  improve  it. 

We  assume  that  we  have  a  large  scale  system  that  cannot  be  exercised  as  one  wants 
and  whose  effectiveness  has  to  be  assessed:  in  order  to  do  so,  one  needs  to  gather  data 
concerning  this  large  scale  system;  indeed,  to  assess  the  effectiveness  with  the  SEA 
methodology,  one  must  determine  the  system  locus  of  the  system  at  hand.  This  requires 
operating  the  system  at  an  extremely  large  number  of  sets  of  parameter  values. 

A  common  approach  to  gather  data  on  such  a  system  is  to  build  a  test-bed  which  is 
assumed  to  be  a  complete  surrogate  of  the  system:  this  test-bed  can  be  exercised  as  one 
wants,  but,  since  it  is  often  expensive  and  time  consumming  to  operate,  only  a  small 
number  of  experiments  can  be  run.  The  procedure  presented  in  this  chapter  provides  a 
means  to  determine  a  small  set  of  experiments  to  be  run  on  the  test-bed:  the  small  number 
of  experimental  values  will  be  combined  with  the  results  obtained  from  a  simplified 
mathematical  model  of  the  system  to  construct  the  system  locus  of  the  test-bed.  The 
underlying  assumption  is  that  the  test-bed  is  an  excellent  approximation  of  the  actual 
system.  Because  of  this  assumption,  no  distinction  will  be  made  in  this  chapter  between 
the  actual  system  and  its  surrogate,  the  test-bed;  the  only  distinction  that  will  be  considered 
is  the  one  between  the  simplified  mathematical  model  and  the  actual  system  (or  the 
test-bed).  After  determining  the  system  locus  of  the  actual  system,  one  can  evaluate  its 
effectiveness. 


3.2  OVERVIEW  OF  THE  PROCEDURE 

As  mentioned  in  the  introduction,  the  effectiveness  of  a  large  scale  system  cannot  be 
evaluated  directly;  indeed,  since  the  actual  system  cannot  be  exercised  as  one  wants,  it  is 
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impossible  to  determine  its  system  locus:  it  would  require  to  run  an  extremely  high  number 
of  experiments. 

In  this  chapter,  a  simplified  mathematical  model  of  the  actual  system  is  considered; 
this  model  can  be  represented  by  a  mapping  "f*  from  the  parameter  space  into  the  MOP 
space.  Two  mappings,  and  consequentely  two  loci  can  be  considered.  First,  using  the 
mathematical  model  "f ’,  the  parameter  space  can  be  mapped  into  the  "model"  system  locus. 
Second,  the  actual  system,  if  it  could  be  exercised  for  all  the  values  of  the  parameter  space, 
would  yield  the  "actual"  system  locus  (Fig.3.1).  Since  the  model  is  a  simplified  one,  the 
model  locus  is  an  approximation  of  the  actual  locus. 


Actual  Locus  Model  Locus 
L  L 


Parameter  Space 


MOP  Space 


Fig.3.1:  Actual  and  model  loci 


This  chapter  aims  at  determining  the  actual  locus  of  the  system;  the  key  idea  is  to 
obtain  the  model  locus,  and  a  few  points  of  the  actual  locus,  and  to  determine  a  mapping 
"T"  that  transforms  the  model  locus  into  the  actual  locus  in  the  MOP  space.  This  mapping 
T  that  goes  from  the  MOP  space  to  the  MOP  space  will  be  obtained  from  the  model  locus 
and  the  few  points  determined  through  experimentation  of  the  actual  locus.  Then,  the  actual 
locus  will  be  obtained  indirectly:  if  "A11  is  the  actual  mapping  from  the  parameter  space  into 
the  MOP  space,  then  A=Tof,  where  "o"  denotes  the  composition  of  two  functions 
(Fig. 3. 2).  With  this  algorithm,  only  a  few  points  belonging  to  the  actual  locus,  and 
therefore  only  a  few  experiments  will  be  necessary  to  evaluate  the  locus  of  the  actual 
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system  and  eventually  the  effectiveness  of  this  system. 


Parameter 

Locus 


Fig.3.2:  Determination  of  the  actual  mapping 

On  Fig.3.2  the  actual  mapping  A  is  denoted  (A  *  Tof)  to  emphasize  the  fact  that  it 
cannot  be  obtained  directly  but  only  as  a  composition  of  f  and  T. 

Since  f  is  assumed  to  be  known,  only  T  has  to  be  evaluated  if  one  wants  to 
determine  the  actual  mapping.  The  focus  of  the  remaining  part  of  this  chapter  is  therefore 
the  determination  of  T. 

The  first  step  is  to  obtain  a  small  number  of  points  in  the  actual  locus.  In  sections 
3.3  and  3.4,  tools  for  selecting  these  points  are  introduced:  they  will  be  used  in  section  3.5 
where  the  actual  locus  will  be  constructed. 

In  the  next  section,  an  inversion  algorithm  that  will  be  used  later  in  this  chapter  for 
selecting  points  in  the  actual  locus  is  presented. 


3.3  INVERSION  ALGORITHM 
3.3.1  Introduction 


In  this  section  an  algorithm  is  presented  that  allows  one  to  determine  a  parameter 
vector  corresponding  to  a  given  MOP  vector  in  the  model  locus.  Later  in  the  development, 
this  alghorithm  will  be  applied  to  a  small  number  of  points  in  the  model  locus  to  obtain  an 


equal  number  of  parameter  vectors.  Experiments  will  be  run  at  these  parameter  values  to 
yield  the  small  number  of  points  belonging  to  the  actual  locus  we  are  looking  for. 

Let  us  suppose  that  we  want  to  reach  a  desired  point  in  the  MOP  space;  the  purpose 
of  the  inversion  algorithm  presented  below  is  to  find  a  value  in  the  parameter  space  that 
yields  through  "f*  the  desired  value  in  the  MOP  space:  it  is  to  invert  the  mathematical 
model  "f*  for  a  given  point  in  the  MOP  space.  The  mapping  "f"  is  not  assumed  to  be 
invertible  for  each  MOP  value;  in  addition,  the  dimension  of  the  parameter  space  and  of  the 
MOP  space  may  be  different:  therefore,  the  inversion  algorithm  must  determine  whether 
”f'  can  be  inverted  at  a  specific  MOP  value,  and  if  it  can,  the  algorithm  will  yield  a 
parameter  vector  (which  may  not  be  unique)  corresponding  to  the  desired  MOP  value. 

3.3.2  Notation 

A  system  with  m  MOPs  {x^li=l,...,m}  and  n  parameters  {pjlj=l,...,n}  is 
considered;  Generally,  n  and  m  are  not  equal.  If "f"  is  the  mapping  from  the  parameter 
space  into  the  MOP  space,  then  x=f(p)  where  x  and  p  are  column  vectors.  If  pq  is  an  initial 
combination  of  parameters,  the  corresponding  point  in  the  MOP  space  is  XQ=f(po);  Pq 
be  called  a  "basic  operating  point". 

3.3.3  The  inverse  problem 

The  next  objective  is  to  determine  the  small  variation  dp  around  pq  coresponding  to  a 
small  variation  di  around  xq.  Since  f  is  generally  non-invertible,  one  has  to  find  an 
algorithm  that  determines  dp-  If  f  is  assumed  to  be  differentiable  at  p  =  pq  ,  then 
2L  -  +  [df(pQ)/dp].dp 

If  £  =  [df(pg)/dp]  then, 

£  =  (gjj)  with  gjj=$Xj(Eo)/3pj],  i=l . m  ;  j=l . n 


where  £  is  a  m  by  n  matrix. 

Singular  value  decomposition  of  G 

The  singular  value  decomposition  of  £  yields: 

£=LLXY’  and  £=£'.£.¥ , 

where  £  is  a  m  by  m  unitary  matrix,  V  a  n  by  n  unitary  matrix,  and  £=(sjj)  an  m  by  n 
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matrix  with  Sjj=0  if  i*j  and  s^=Sj>0  .  The  Sj  are  the  singular  values  of  Q.  If  Qij)  are  the 
column  vectors  of  matrix  li  and  (Vp  the  column  vectors  of  matrix  Y»  then 

Q.V^-Ui . 

Moreover,  (Vpi=l . n  is  a  basis  of  Rn  and  (Uj),j=l,...,m,  is  a  basis  of  Rm. 

Application 

If  one  considers  a  small  variation  A  in  the  MOP  space.one  can  decompose  A  as 
follows: 


A=D1iI1+D2U2+-.+DmUm  • 


If  Sj*0  for  i=l,...,m  then  a  corresponding  direction  in  the  parameter  space  is 
d=(D1/s1)Yi-KD2/s2)Y2+...+(Dm/sm)Y 

This  direction  may  not  be  unique  if  n>m:  Indeed,  if  n  >  m,  d+Vj  n  >i>  m  satisfies  the 
problem,  because  Q.Vj=0  for  n  >i>  m. 

If  some  singular  values  are  equal  to  zero,  it  means  that  there  are  points  unreachable 
from  xq. 

Therefore  if  Sj,  i=l,...,m,  are  the  singular  values  of  A  and  if  Sj*0  for  i=l...k  (k<m) 
and  Sj=0  for  i=k+l...m  then: 

•For  any  direction  in  the  MOP  space  such  that 


A=D1U1+D2U2+-+Dki4 


(3.1) 


one  can  find  a  corresponding  direction  in  the  parameter  space: 


d=(Dj/s  ^  )Vj+(D2/s2)  V2+...+(Djc/sjc)Y[c. 


(3.2) 


(Since  sj  for  i=l...k  is  not  equal  to  zero,  one  can  divide  by  sp. 
•For  any  direction 


A=Dk+1iIk+1+Dk+2nk+2+....+DmLLrn 


with  one  of  the  Dj  (i=k+l,....,m)  not  equal  to  zero,  it  is  impossible  to  find  a  corresponding 
direction  in  the  parameter  space.(Indeed,  if  for  example  sjc+^=0,  it  means  that  is  not 
element  of  the  image  space  of  the  paramater  space.) 


Therefore,  the  singular  value  decomposition  of  the  matrix  Q  provides  a  means  to 
find  a  small  variation  in  the  parameter  space  that  corresponds  to  the  desired  variation  in  the 
MOP  space.  It  may  be  impossible  to  find  such  a  direction  if  the  desired  direction  in  the 
MOP  space  is  not  element  of  the  image  space  of  the  parameter  space  through  the 
algorithm  tells  us  whether  it  is  possible  or  not.  In  general,  when  it  is  possible  to  find  a 
corresponding  direction  in  the  parameter  space,  this  direction  is  not  unique;  in  this  case, 
there  arc  some  degrees  of  freedom  in  choosing  this  direction  d-  For  example,  one  can  use 
the  smallest  norm  of  the  vector  obtained  in  the  algorithm,  or  the  smallest  number  of 
parameters  affected  in  the  parameter  space,  as  rules  for  selecting  the  direction  d-  In  what 
follows,  we  will  always  choose  the  direction  determined  by  the  vector  d  whose  euclidian 


■Id"  =  [(di)2  +  (d2)2  +  ...  +  (dm)2  ]1/2  (33) 

is  the  smallest  indeed,  since  we  are  dealing  with  a  linearized  function,  a  large  norm  d  will 
degrade  the  accuracy  of  the  algorithm  if  the  gradient  matrix  A  is  ill-conditioned;  another 
reason  for  choosing  this  direction  is  that  we  want  the  new  parameter  value  to  be  as  close  as 
possible  to  the  initial  one:  it  corresponds  to  the  slightest  variations  of  the  operating 
conditions  of  the  system. 

3.3.4  Single  stage  parameter  value  determination 


With  the  notation  defined  above,  it  is  possible  to  explore  the  system  locus  around 
2Sq;  If  for  instance,  one  wants  to  reach  point  x<i  (for  x-desired)  close  to  xq  in  the  MOP 
space,  one  can  apply  the  procedure  described  above  to  d=Xd"£(>  The  outcome  is  a  vector 
d=a'-Eo  in  Ac  parameter  space;  Therefore,  the  parameter  value  corresponding  to  is 
B'=d+Bo-  As  stressed  above,  some  points  close  to  xq  may  he  unreachable  because  of 
singular  values  equal  to  zero.  Locally,  around  xq,  it  is  therefore  possible  to  explore  the 
system  locus  of  the  model  in  a  directed  manner.  An  interesting  point  is  that  to  go  from  xq 
to  Xd  only  m  directions  are  involved  in  the  parameter  space. 

If  one  wants  to  reach  Xd  close  to  xq  in  the  MOP  space,  the  new  operating  point  g' 
can  be  computed  with  the  mathematical  model  and  the  procedure  described  above.  Since  it 


is  not  guaranteed  that  the  &'  computed  in  the  singular  value  algorithm  will  belong  to  the 
parameter  locus  (Fig.3.3),  one  must  consider  the  projection  of  p'  on  the  parameter  locus. 
In  what  follows  p=proj(p')  refers  to  the  projection  of  p'  on  the  parameter  locus. 

Because  of  the  projection  p  of  p'  on  the  parameter  locus,  and  because  of  the 
linearized  model  used  in  the  inversion  algorithm,  one  cannot  expect  that  the  simplified 
model  f  applied  to  the  new  parameter  value  p  will  yield  the  desired  point  in  the  MOP  space 
(Fig.3.3);  in  general: 

f(p)  =  s'  *  Xd  (desired  value  in  the  MOP  space)  (3.4) 
because  of  linearization  and  because  of  the  projection  of  p'  on  the  parameter  locus. 


Error  -  £  -  i (j 


Fig.3.3:  Determination  of  p,  and  error  in  the  MOP  space 

3.3.5  Multi-stage  parameter  determination 

As  we  have  just  seen,  the  single  step  parameter  determination  generally  yields  an 
inaccurate  approximation  in  the  MOP  space.  If  one  is  interested  in  an  accurate 
determination,  one  can  go  through  the  inversion  algorithm  again  and  again,  until  the  error 
vector  is  small  enough  for  the  application  at  hand.  For  the  n^1  stage  of  the  process,  the 
initial  parameter  value  is  the  one  that  has  been  computed  at  stage  n-l;  the  mapping  f  is 
linearized  at  that  point  and  the  single  stage  inversion  algorithm  is  applied. 

The  process  stops  when  the  MOP  point  obtained  by  applying  f  to  the  computed 


parameter  value  is  close  enough  (for  the  application  considered)  to  the  desired  point  in  the 
MOP  space.  This  multi-stage  process  is  represented  as  a  flowchart  in  Fig. 3. 4. 
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Fig.3.4:  Multi-stage  parameter  determination 


Therefore,  this  inversion  algorithm  provides  a  means  to  find  a  parameter  value  such 
that  the  simplified  mathematical  model  "f  ’  when  applied  to  this  parameter  value  will  yield  a 
point  in  the  MOP  space  arbitrary  close  to  a  desired  MOP  value. 
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3.4  EXPERIMENTAL  DESIGN 


3.4.1  Introduction 

After  applying  the  inversion  algorithm,  we  have  determined  p  such  that:  Xd=f(p);  the 
error  in  the  determination  of  p  is  assumed  to  be  so  small  as  to  be  negligible.  At  the  basic 
operating  point  considered  we  have  So=f(Po)- 

One  can  assume  that  the  simplified  model  has  been  calibrated  at  the  basic  operating 
point:  a  reasonable  asssumption  is  that  the  simplified  model  and  the  actual  system  coincide 
atp=po: 

25q  =*  f(po)  *  Actual  System  (pg). 

To  determine  what  the  actual  system  locus  looks  like,  we  will  choose  a  small 
number  of  desired  points  in  the  MOP  space  (x^);  how  these  points  are  chosen  is  described 
in  section  3.4.2.  For  each  of  these  point,  with  the  algorithm  presented  in  section  3,  we  will 
find  a  parameter  vector  such  that  *  f(p).  Then,  an  experiment  will  be  run  on  the  actual 
system  at  this  parameter  vector  the  outcome  of  the  experiment  will  be  a  point  in  the 
MOP  space.  Since  the  simplified  model  and  the  actual  system  are  close  but  not  equal,  the 
values  *e  and  are  going  to  be  different  (Fig.3.5). 


desired  £d  inversion^  p  experiment  ^  * 
of  "f"  e 


Fig.3.5:  Mathematical  and  Experimental  MOP  values 
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The  focus  of  this  section  is  the  determination  of  the  points  2C4  in  the  model  locus;  for 
each  of  the  points  selected  in  this  section,  we  will  apply  the  inversion  algorithm  described 
above;  it  will  yield  as  many  parameter  vectors  as  points  x^;  for  each  parameter  vector 
computed  that  way,  the  corresponding  experiment  will  be  run  and  yield  an  experimental 
MOP  vector  2^  belonging  to  the  actual  locus.  With  this  procedure,  for  each  desired  point 
selected  in  the  model  locus,  there  will  be  a  corresponding  point  in  the  actual  locus;  to  a 
given  choice  of  points  in  the  model  locus  will  correspond  a  selection  of  points  in  the  actual 
locus. 

3.4.2  Selection  of  points  in  the  MOP  space 

The  first  step  that  has  to  be  completed  in  the  experimental  design  process  presented 
above  is  the  selection  of  desired  values  in  the  MOP  space:  one  must  select  a  small  number 
of  vectors  2^4-  These  vectors  2S4  will  be  denoted  as  an  indexed  family  (2^)  i=l,...,r.  The 
number  of  points  selected  will  be  exactly  the  number  of  experiments  that  will  be  run  on  the 
actual  system. 

The  only  available  information  for  selecting  the  points  2*4  is  the  system  locus 
obtained  from  the  mathematical  model.  With  the  mapping  f,  one  can  easily  determine  this 
locus.  Since  one  is  interested  in  having  a  selection  of  points  that  represents  the  entire  locus 
as  opposed  to  any  of  its  region,  the  points  that  will  be  chosen  must  be  distributed  all  over 
the  locus.  A  simple  way  to  choose  a  small  number  of  points  that  are  representative  of  the 
locus  is  to  inscribe  it  in  an  n-dimensional  rectangular  parallilepiped,  and  to  choose  the 
tangeancy  points;  the  two  dimensional  case  is  shown  in  Fig.3.6.  This  method  for  choosing 
the  points  in  the  model  locus  is  not  only  simple;  it  is  the  one  that  allows  one  to  select  the 
minimum  number  of  points  for  looking  at  the  whole  locus. 

The  vectors  J&4  that  we  are  going  to  consider  are  the  points  where  the  system  locus 
and  the  parallilepiped  determined  above  are  tangent  (Fig.3.6).  We  will  denote  r  the  number 
of  these  vectors  2S4;  in  each  dimension,  they  correspond  to  the  maxima  and  minima  of  each 


parallilepiped 


Rg.3.6:  Determination  of  the  2^ 


In  general,  more  than  one  point  in  the  system  locus  will  be  tangent  to  the 
parrallilepiped  defined  above:  for  example,  the  system  locus  itself  may  be  a  parallilepiped, 
or  part  of  the  system  locus  boundary  may  be  a  plane  surface  included  in  one  of  the  surface 
that  defines  the  parallilepiped  (Fig.3.7);  there  will  be  an  infinite  number  of  tangency  points 
in  these  two  latter  situations.  The  number  of  tangency  points  may  also  be  finite  (Fig.3.7). 
Since  we  want  to  choose  only  a  small  number  of  points,  and  since  we  are  interested 
ultimately  in  determining  how  the  parallilepiped  is  deformed  when  one  goes  from  the 
mathematical  locus  to  the  actual  one,  we  will  consider  only  one  point  of  tangency  for  each 
surface  defining  the  parallilepiped.  This  point  will  be  the  center  of  gravity  of  all  the 
tangency  points  for  each  surface  of  the  parallilepiped;  if  the  number  of  points  is  finite,  the 
center  of  gravity  may  not  belong  to  the  locus  (the  selected  points  must  belong  to  the  model 
locus  if  one  wants  to  be  able  to  find  a  corresponding  parameter  value):  in  that  case,  the 
closest  point  of  the  locus  to  the  center  of  gravity  will  be  chosen.  Given  the  constraint  that 
one  must  look  at  the  whole  locus,  the  procedure  presented  above  selects  the  minimum 
number  of  points;  it  will  eventually  corresponds  to  the  minimum  number  of  experiments 
that  needs  to  be  run. 

Therefore,  we  will  select  2n  (i.e.  r=2n)  points  on  the  mathematical  locus,  where  n  is 
the  dimension  of  the  MOP  space. 


parallilepiped 


locus 


parallilepiped 


Fig. 3.7:  Three  system  locus  configurations 


3.5  RECONSTITUTION  OF  THE  ACTUAL  SYSTEM  LOCUS 


3.5.1  Introduction 


We  now  have  all  the  tools  required  to  select  a  small  number  of  points  in  the  actual 
locus,  and  to  compute  the  mapping  T  that  transforms  the  model  locus  into  the  actual  locus. 
First,  let  us  determine  the  small  number  of  points  we  axe  looking  for  in  the  actual  locus. 


3.5.2  Points  in  the  actual  locus 


We  apply  the  inversion  algorithm  presented  in  section  3.3  for  the  r  points  (x^) 
i=l,...jr  selected  in  section  3.4;  it  yields  r  vectors  (j>j)  i=l,...,r  in  the  parameter  space. 
Then,  experiments  corresponding  to  these  parameter  vectors  are  run  on  the  actual  system: 
that  is,  the  experimental  conditions  are  set  as  required  by  the  parameter  vectors.  This  is 
always  feasible  because  the  parameter  vectors  determined  with  the  inversion  algorithm  are 
constrained  to  belong  to  the  set  of  admissible  parameters  (parameter  locus):  indeed,  they 
are  projected  on  the  parameter  locus.  As  shown  in  section  3.4,  the  procedure  for  choosing 
the  points  x^j  is  the  one  that  allows  one  to  look  at  the  whole  locus  with  the  mimmun 
number  of  points:  thus,  given  the  contraint  that  one  must  look  at  the  whole  locus,  the 
minimum  number  of  experiments  are  actualy  run.  The  outcome  of  these  experiments  will 
be  r  experimental  values  (x^j)  i=l,...,r  in  the  MOP  space  (Fig.3.8). 
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Fig.3.8:  Experimental  results 

To  the  r  experimental  values,  one  must  add  because  an  assumption  of  the 
procedure  is  that  the  simplified  model  and  the  actual  system  coincide  at  In  general,  we 
have  therefore  r=2n+l  experimental  vectors  with  n  coordinate  for  each.  On  the  basis  of 
these  results  we  must  determine  the  system  locus  of  the  actual  system. 

Therefore,  with  this  procedure,  a  small  number  of  experiments  have  been  run  to 
yield  a  few  (2n+l)  points  in  the  actual  locus.  On  the  basis  of  this  small  number  of  points 
and  the  model  f,  we  will  determine  the  transformation  T  introduced  in  section  3.2  that 
maps  the  model  locus  into  the  actual  locus. 

3.5.3  From  the  model  to  the  actual  locus 


We  have  only  2n+l  points  in  the  actual  locus,  where  n  is  the  dimension  of  the  MOP 
space.  The  focus  of  this  section  is  to  determine,  for  each  single  point  Xj^  in  the  model 
locus,  the  corresponding  point  &a  in  the  actual  locus:  it  is  the  computation  of  a 
transformation  T  that  maps  the  model  locus  into  the  actual  locus.  For  each  xm  in  the  model 
locus,  we  will  have  xa  =  T(xm). 

We  assume  that  the  transformation  from  the  model  locus  to  the  actual  one  is  the 
composition  of  a  translation  V  and  a  linear  transfer  nation  L;  thus,  for  any  vector  xm  in  the 
mathematical  locus,  the  corresponding  point  in  the  actual  locus  will  be  xa  with 

£a  =  Lxm  +  X  .  (3.5) 


37 


.  J 


Such  a  transformation  is  described  by  n^+n  coefficients  in  an  n-dimensional  space  (square 
matrix  L  with  n  rows  and  n  columns  and  vector  V).  To  determine  this  transformation,  we 
have  m  equations  where  r  is  the  number  of  experiments:  since  for  r=2n+l, 

m  =  2n^  +  n  >  n^  +  n  , 

one  can  implement  a  least  square  procedure  to  compute  L  and  V. 

Letei  =  IILxdi  +  V-)ceill  (3.6) 

where  II  II  represents  the  euclidian  norm  (see  equation  3.3).  The  number  ej  is  the  norm  of 
the  difference  between  the  actual  locus  vector  obtained  by  the  transformation  and  the 
experimental  value.  Then  the  transformation  (3.4)  is  the  one  that  minimizes  the  following 
expression  (r  is  the  number  of  experiments). 

J  =  ej+e2+  ...  +er  (3.7) 

With  this  transformation  T,  for  each  point  xm  in  the  model  locus,  the  corresponding 
point  in  the  actual  locus  is 

*a  =  Ttem)-  (3.8) 

This  "T"  is  the  same  as  the  one  introduced  in  section  3.2  (Fig.3.2):  therefore,  we 
can  interpret  A  as  the  transformation  Tof  that  maps  the  parameter  locus  into  the  actual 
system  locus. 

3.5.4  Conclusion 

In  this  section  the  tools  introduced  in  sections  3.3  and  3.4  have  been  used  to 
determine  the  mapping  R=Tof  from  the  parameter  locus  into  the  actual  locus.  This  mapping 
is  the  outcome  of  a  mathematical  model  combined  with  the  small  number  of  experimental 
values  that  one  can  usually  afford  to  run  on  the  actual  system.  It  should  be  noted  at  that 
point  that,  if  the  actual  system  could  be  exercised  for  all  the  values  of  the  parameter  locus, 
it  would  probably  yield  an  actual  system  locus  snghtly  different  from  the  one  obtained  at 
the  end  of  this  section;  given  the  experimental  constraints,  the  actual  locus  obtained  with 


A=Tof  is  the  best  approximation  of  the  actual  system  locus  that  would  be  obtained  by 
running  as  many  experiments  as  one  would  like. 


3.6  STEPS  OF  THE  PROCEDURE 

The  steps  of  the  experiment  design  procedure  and  their  interrelationships  are  shown 
in  Fig.3.9  and  are  summarized  below: 

Step  1:  Define  a  mathematical  model  "f”  of  the  system  at  hand. 

Step  2:  Analyze  this  model  with  the  System  Effectiveness  Analysis 

methodology  in  order  to  obtain  a  mathematical  system  locus. 

Step  3:  Determine  r  points  of  interest  in  the  mathematical  locus 

(tangency  points  with  a  n-dimensional  parallilepiped). 

Step  4:  With  the  inversion  algorithm,  for  each  find  a  parameter 

vector  in  the  parameter  locus  such  that  2S<ji=f(£!<ii)- 

Step  5:  For  each  run  an  experiment  on  the  actual  system:  it  yields 

r  experimental  values  ^j. 

Step  6:  To  the  r  mathematical  MOP  points  (i^i)  and  their 

corresponding  experimental  points  Q^j),  apply  the  least 
square  procedure  to  find  a  transformation  T  that  maps  the 
mathematical  locus  into  the  actual  locus. 

Step  7:  Apply  A  =  Tof  to  each  point  in  the  parameter  locus  to  obtain  the 

actual  system  locus. 

Then,  with  the  actual  system  locus,  one  can  evaluate  the  effectiveness  of  the 
actualsystem,  as  opposed  to  evaluating  the  effectiveness  of  a  simplified  model. 
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Fig.3.9:  Steps  of  the  experiment  design  procedure 


3.7  CONCLUSION 

In  this  chapter,  an  algorithm  to  design  experiments,  and  to  evaluate  the  system 
locus  of  a  actual  system  on  the  basis  of  a  simplified  model,  has  been  presented.  As  shown 
in  the  development  of  the  chapter  the  implementation  of  the  selected  experiments  is  always 
possible.  Moreover,  by  inscribing  the  model  locus  into  a  parallilepiped,  the  minimum 
number  of  experiments  that  needs  to  be  run  has  been  obtained:  this  is  a  deliberate  choice 
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that  allows  one  to  look  at  the  whole  locus  by  running  as  few  experiments  as  possible.  Such 
an  algorithm  combined  with  the  System  Effectiveness  Analysis  methodology  enables  one 
to  determine  the  effectiveness  of  an  actual  system  as  opposed  to  making  the  evaluation  on  a 
model  of  this  system. 

After  having  evaluated  the  effectiveness  of  an  actual  system,  one  may  be  interested 
in  improving  it;  the  purpose  of  the  following  chapter  is  the  presentation  of  a  dynamic 
programming  algorithm  designed  to  improve  a  system  over  time. 


CHAPTER  4 

SYSTEM  IMPROVEMENT  AND  DYNAMIC  PROGRAMMING 


4.1  INTRODUCTION 

Since  the  development  of  a  large  scale  system  lasts  generally  many  years,  the 
developers  need  tools  to  plan  the  evolution  of  the  system  at  hand;  because  of  budget  and 
time  constraints  the  system  is  built  progressively,  and  components  are  added  gradualy: 
therefore,  the  future  modifications  of  a  system  are  known  by  the  system  developer  for  a 
given  time  horizon.  During  its  lifetime,  the  system  will  be  modified  at  given  time  intervals 
(generally  one  year  intervals  for  administrative  reasons):  therefore,  the  number  of  stages 
the  system  is  going  to  go  through  can  be  estimated  accurately.  During  the  evolution 
process,  the  goal  pursued  by  the  system  developer  is  the  maximization  of  the  effectiveness 
over  the  system  lifetime. 

In  order  to  carry  out  this  maximization,  the  sytem  developer  must  determine  the 
optimal  sequence  of  modifications.  The  new  system  will  be  represented  in  the  MOP  space 
by  a  new  locus,  and  will  correspond  to  a  given  number  of  modifications:  add  or  remove 
such  and  such  components,  change  a  particular  structure,  and  so  on.  Since  all  the  changes 
cannot  be  carried  out  at  the  same  time,  the  system  will  evolve,  in  the  sense  defined  in 
chapter  2,  as  components  get  added  or  removed.  The  problem  is  to  optimize  the  multi-stage 
decision  process  regarding  the  implementation  of  these  modifications.  In  this  chapter,  we 
are  going  to  present  an  algorithm  that  enables  one  to  determine  under  certain  assumptions 
an  optimal  sequence  of  modifications  to  improve  an  existing  system  over  time.  Since  one  is 
interested  in  optimizing  the  effectiveness  of  the  actual  system  as  it  evolves,  the  algorithm 
presented  in  chapter  3  can  be  used  to  determine  the  effectiveness  of  the  actual  system  on 
the  basis  of  a  simplified  mathematical  model  at  each  stage. 

Because  of  the  features  of  an  evolution  process  such  as  the  one  outlined  above,  the 
basic  assumptions  underlying  this  chapter  are  as  follows:  first  of  all,  we  assume  that  the 
possible  modifications  of  the  system  arc  known  and  are  constrained  to  be  in  a  given  set; 
moreover,  the  number  of  modifications  that  the  initial  system  is  going  to  go  through  is 
assumed  to  be  known.  The  second  assumption  is  that  the  goal  of  the  system  developer  as 
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time  goes  by  is  the  maximization  of  an  objective  function  which  is  a  linear  combination  of 
the  MOEs  of  the  system  at  different  stages  of  its  evolution. 


4.2  MODELING  THE  IMPROVEMENT  PROCESS 
4.2.1  Single  stage  evolution 

Before  going  into  a  sequence  of  modifications,  let  us  model  a  single  stage 
evolution  for  a  system  (Fig.4.1). 

Let  Xq  be  a  given  system  which  does  not  evolve  between  time  tQ  and  tj;  MOEq  is 
the  effectiveness  of  Xq.  If  at  time  t]  system  Xq  evolves  because  of  an  exogenous 
modification  Uq,  the  new  system  is  denoted  Xj.  A  modification  Uq  is  a  decision  variable. 


Initial 

system 


New 

System 


u  Decision 
0  variable 


Effectiveness 
of  the  new 
system 


Fig.4.1:  Single  Step  system  evolution 

To  the  new  system  X]  is  associated  a  measure  of  effectiveness  MOEj.  One  can 
write  the  following  evolution  equations: 

X1=Fq(X0,Uo)  ,  and  (4.1) 

MOEj^MjCX^  (4.2) 

where  Fq  is  the  function  that  describes  the  transformation  process,  and  Mj  the  function 
determining  the  MOE  for  X  j.  The  process  sketched  out  above  may  be  viewed  as  a  single 
stage  evolution. 
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Fig.4.2:  Multiple  stage  evolution 

For  t  e  [ti.tj+i),  the  system  is  non-evolving:  for  each  i,  tj  is  defined  as  being  the 
time  when  the  system  evolves  in  the  sense  of  chapter  2.  For  t  £  [tj.,4+1),  Xj(t)=Xj,  and  the 
following  relations  can  be  deduced  from  the  previous  subsection  (equations  4. 1  and  4.2): 

Xi=Fi_ i (xi_  i »ui_ i ) .  and  (4.3) 

MOEi=Mi(Xi)  for  i=l ...n .  (4.4) 

The  functions  Fj  are  fixed  because  a  given  initial  state  and  a  given  decision  yield  a 
known  outcome.  The  MOEj  are  global  measures  of  effectiveness;  They  are  functions  of 
partial  MOE’s: 


MOEi(Xi)=ui[E1(Xi),E2(Xi),...£p(Xi)]=Mi(Xi),  (4.5) 

where  Ej,...,Ep  are  partial  MOEs.  As  the  modification  process  goes  on,  one  may  add  new 
partial  MOEs  or  consider  a  new  aggregation  of  the  existing  partial  MOEs:  In  general  Uj*uj, 
and  Mj#Mj  for  i*j. 

The  overall  goal  of  the  modification  process  is  to  maximize 
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V=ajMj(Xj)+a2M2(X2)+—+anMn(Xn) 


(4.6) 


where  a^...  a,^  are  weighting  coefficients,  where  n  is  given,  and  where  the  decision 
variables  are  constrained  to  be  in  a  specific  set.  The  initial  system  is  given:  One  must  find 
the  sequence  of  modification  that  maximizes  V. 


4.3  DYNAMIC  PROGRAMMING 

In  this  section,  a  dynamic  programming  algorithm  is  applied  to  the  multistage 
decision  process  presented  above. 

4.3.1  First  step 

Suppose  that  the  (n-l)-stage  process  (Xj^2»*"»^n-P  ^as  *5ccn  optimized;  Then, 
Xn_j  is  given  and  the  maximization  of  V  is  equivalent  to  the  maximization  of  anMn(Xn). 
Therefore,  the  problem  is  reduced  to  a  single  maximization.  Given  Xn_j,  find  Un_j  from 
the  given  set  of  decisions  such  that: 


vn-l(Xn-l)=MaxUn.1(anMn(Xn)}  = 

MaxUn.  j  { znMn[FnA(Xn_  { ,Un_  {))).  (4.7) 

To  solve  this  problem,  one  can  apply  the  methodology  developed  by  Karam 
[1985]  for  a  single  stage  process:  indeed,  Karam  considered  a  single  stage  process  with  a 
decision  variable  constrained  to  be  in  a  specific  set:  for  each  stage,  we  have  to  solve  a 
problem  of  that  kind. 

4.3.2  Remaining  steps 

Suppose  that  the  (n-k)-stage  process  (X|X2»—rXn.jc)  has  been  optimized;  Then 
Xn_k  is  fixed  and  the  problem  becomes:  find  Un_k  in  the  given  set  of  decisions  such  that 


vn-k^Xn-k^_^ax^n-k  ( ® ^n-k)+vn-k+  \  (Xn-k+ 1 )} 
with  B(Un.k)=an.k+1  Mn.k+1  [Fn.k(Xn.k,Un.k)] 

^  Xn-k+l=*Vk(xn-k’Un-k) 


(4.8) 

(4.9) 

(4.10) 


Once  again  it  is  a  single  variable  maximization  problem,  and  one  can  apply  the 
methodology  developed  by  Karam. 


4.3.3  Summary 

The  n  variable  maximization  process  is  reduced  to  n  single  variable  problems.  By 
setting  i=n-k,  one  can  write: 

vi(Xi)=MaxUi  { B(U  i)+vi+ j  (Xi+  j ) }  (4.11) 

with  B(Ui)=ai+1Mi+1[Fi(Xi,Ui)]  (4. 12) 

and  X^+j*Fj(Xj,Uj)  i=0,...,n-l  and  Xq  fixed.  (4.13) 

The  solution  of  this  problem  yields  the  optimal  sequence  of  decision  variables 
Uj,U2,....Un,  each  decision  being  element  of  a  given  set 


4.4  DYNAMIC  PROGRAMMING  AS  PATH  DETERMINATION 

In  this  section  we  present  a  graphical  interpretation  of  the  algorithm  developed 
above;  to  do  so  we  will  apply  the  procedure  to  a  simple  example. 

Let  us  consider  a  three  stages  evolution  process  (n=3)  for  an  initial  system  Xq;  the 
decision  is  constrained  to  be  either  uj  or  U2-  We  assume  that  the  goal  is  the  maximization 
of  V=MOE  j  +MOE2+MOE3 . 

The  dynamic  programming  algorithm  can  be  represented  as  a  tree  (Fig.4.3)  where 
each  node  corresponds  to  a  given  state  of  the  system,  and  each  branch  corresponds  to  a 
particular  decision. 

In  Fig.4.3,  the  values  of  the  MOEs  for  each  state  of  the  system  are  shown  on  the 
branches;  in  this  figure,  for  each  node,  the  upper  branch  corresponds  to  decision  uj  and 
the  lower  branch  corresponds  to  decision  U2-  To  find  the  optimal  path  on  Fig.4.3,  we  start 
from  X2  and  choose  the  path  leading  to  a  state  X3  with  the  maximal  MOE:  we  do  that  for 
each  initial  X2.  Then,  to  determine  the  best  decision  to  go  from  Xj  to  X2  we  choose  the 
path  such  that  MOE3+MOE2  is  maximal  along  that  path.  Finally,  to  determine  the  best 
decision  to  go  from  Xj  to  X2,  we  choose  the  path  such  that  MOE3+MOE2+MOE1  is 
maximal  along  that  path.  The  optimal  sequence  of  decision  is,  for  the  example  of  Fig.4.3: 
u  1 ,  u  1 ,  u2;  it  corresponds  to  the  maximal  value  for  the  MOE  sum:  1.9 


Fig.4.3:  A  simple  example  of  dynamic  programming 

In  the  example  considered  above,  one  sh<puld  note  that  after  choosing  decision  u  j 
to  go  from  Xq  to  Xj,  it  is  optimal  to  choose  uj  and  not  112  as  the  second  decision, 
although  the  effectiveness  coorresponding  to  uj  (0.4)  is  lower  than  the  one  corresponding 
to  U2  (0.5):  the  reason  for  this  is  that  by  accepting  a  temporary  degradation  of  the  system, 
one  can  reach  a  high  effectiveness  state  after  the  third  decision;  this  high  effectiveness  state 
(0.9)  makes  up  for  the  temporary  loss  at  stage  2. 


4.5  CONCLUSION 


In  this  chapter  a  dynamic  programming  algorithm  has  been  presented:  it  allows 
one  to  determine  an  optimal  sequence  of  improvement  for  an  evolving  system.  This 
algorithm  will  be  applied  to  a  real  example  in  this  thesis;  the  focus  of  the  next  chapter  is  the 
presentation  of  this  example. 


CHAPTER  5 

PRESENTATION  OF  AN  AIR  DEFENSE  SYSTEM 


5.1  INTRODUCTION 

In  this  chapter,  a  large  scale  system  is  presented;  it  will  be  used  later  as  an  example 
to  illustrate  the  general  approach  of  the  last  three  chapters.  This  illustration  will  be  provided 
by  a  military  air  defense  system  known  as  "Identification  Friend  Foe  Neutral"  (IFFN). 


5.2  GENERAL  BACKGROUND 

During  the  I970's  studies  conducted  by  NATO  members  and  the  US  Defense 
Science  Board  (DSB)  diagnosed  problems  that  existed  with  the  aircraft  identification  (ID) 
process  within  NATO.  It  stimulated  research  linked  to  the  ID  process  in  real  time  combat 
situations;  One  of  the  main  result  was  to  warn  the  Air  Defense  community  against  relying 
on  a  single  means  for  ID,  and  to  emphasize  the  importance  of  relying  upon  many  systems 
to  distinguish  true  friends  and  neutrals  from  true  foes.  Indeed,  in  modem  warfare,  because 
of  the  extended  ranges  of  missiles,  it  appears  that  the  reliance  on  indirect  identification  is 
crucial.  With  this  new  doctrine,  a  firing  unit  may  be  assigned  to  a  specific  target  without 
having  ever  seen  this  target.  It  raises  a  new  problem  known  as  the  "Indirect  ID"  problem: 
The  identification  process  must  be  reliable  enough  to  prevent  too  many  friends  or  neutrals 
to  be  engaged  and  eventually  killed. 

To  assess  the  effectiveness  of  the  indirect  ID  process,  a  test-bed  was  started  in 
1979.  This  project  is  known  as  the  "Identification  Friend  Foe  or  Neutral  Joint  Test  and 
Evaluation  program"  (IFFN  JT&E)  [IFFN  Test  Plan  1985].  The  purpose  of  the  IFFN 
JT&E  is  "to  assess  baseline  capabilities  within  the  air  defense  structure  to  perform  the 
IFFN  function,  to  identify  deficiencies  in  the  performance  of  that  function,  and  to  define 
potential  near-term  procedural  and  equipment  modifications  for  further  testing". 

Therefore,  the  test-bed  is  a  complete  surrogate  of  the  actual  system  which,  of 
course,  can  hardly  be  tested.  By  conducting  experiments  on  the  IFFN  JT&E  facilities  it  is 
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expected  that  the  effectiveness  of  the  actual  system  can  be  assessed.  Then,  the  test-bed 
should  also  provide  a  means  to  improve  the  real  system  by  adding  or  changing 
components,  structures,  or  doctrines  and  to  evaluate  the  resulting  effectiveness.  In  what 
follows  no  distinction  will  be  made  between  the  test-bed  and  the  real  system:  indeed,  it  is 
assumed  that  the  test  facility  is  an  excellent  model  of  the  actual  system. 

As  one  should  note,  the  approach  followed  in  the  development  of  this  large  scale 
system  fits  exactly  in  the  framework  of  chapter  3  where  a  methodology  to  assess  the 
effectiveness  of  real  systems  has  been  presented.  In  the  remaining  of  this  chapter,  the 
actual  IFFN  system  and  a  simplified  mathenatical  model  of  this  system  are  presented. 

In  the  next  section  the  actual  system  is  presented;  then,  a  simplified  mathematical 
model  will  be  introduced. 


5.3  DESCRIPTION  OF  THE  ACTUAL  SYSTEM 

The  actual  system  we  will  focus  on  is  an  air-defense  system  operating  in  the 
Central  region  of  Europe.  The  overall  mission  of  the  system  is  to  defend  a  specified 
airspace  from  an  air  attack  carried  out  by  enemy  aircraft  and  missiles. 

This  system  is  shown  in  Fig.5. 1;  it  is  composed  of  C2  nodes  that  coordinate  the 
action  of  weapons;  these  nodes  are  represented  in  Fig.5. 1,  and  can  be  classified  into  three 
major  groups:  Fire  Directing  Centers  (FDC),  Control  Reporting  Centers  (CRC),  and  a 
third  group  composed  of  more  specific  nodes  such  as  databases  and  higher  level  nodes. 
Their  role  is  to  coordinate  the  weapons  shown  in  Fig.5. 1;  these  weapons  can  be  divided 
into  two  categories:  Surface  to  Air  Missiles  (SAM)  such  as  Hawks  and  Patriots  units,  and 
fighter  such  as  F-15  and  F-16  aircraft. 

The  mission  of  the  system  is  to  engage  and  destroy  hostile  airborne  targets  or 
otherwise  deny  the  enemy  access  to  the  defended  airspace:  in  particular,  the  enemy  aircraft 
must  be  stopped  before  they  can  fire  missiles  at  friendly  assets.  The  system  must  be 
selective  enough  to  minimize  killing  friends  (F-15  and  F-16)  or  neutrals  such  as 
commercial  aircraft  that  are  assumed  to  be  flying  in  the  Central  Region  at  the  time  of  the 
battle.  To  accomplish  this  mission,  the  air-defense  system  must  perform  a  number  of 
sequential  functions  which  define  the  air  defense  process;  this  process  is  characterized  by 
six  primary  phases:  detection,  tracking,  identification  (ID),  allocation,  engagement,  and 
target  kill. 
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Fig.5.1:  Structure  of  the  actual  IFFN  system 

Abrcviations: 

NE-3A:  NATO  airborne  early  warning  Sytem  (Boeing  707);  this  is  a  high  altitude  detection 
aircraft 

SIS:  Special  Information  System;  this  is  a  source  of  intelligence  information  available  to 
basic  nodes  of  the  system  (CRC  or  Control  and  Reporting  Center  nodes). 

CRC:  Control  and  Reporting  Center;  this  C^  node  is  responsible  for  the  overall 
coordination  of  the  system. 

FDC:  Fire  Directing  Center,  these  C^  nodes  are  responsible  for  the  coordination  of  a 
battalion. 

FU:  Firing  Unit 
BN:  Battalion 
BDE:  Brigade 

In  this  complex  system,  the  missiles  fired  either  by  a  fighter  or  by  a  SAM  unit  are 
Beyond  Visual  Range  (BVR)  weapons:  a  firing  unit  does  not  see  the  targets  it  is  shooting 
at;  the  fire  parameters  are  given  to  this  firing  unit  by  the  C^  system  on  the  basis  of 
identification  performed  by  other  units  called  "detecting  units".  This  indirect  identification 
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process  justifies  the  structure  that  lies  above  the  weapons  in  Fig.5.1:  it  is  the 
responsibility  of  this  structure  to  pass  correct  and  accurate  parameters  to  firing  units. 

In  the  next  section,  a  simplified  model  of  this  complex  system  is  introduced;  the 
remaining  of  the  chapter  will  develop  this  simplified  model. 


5.4  SIMPLIFIED  MODEL:  AN  OVERVIEW 

In  the  simplified  model,  we  are  going  to  focus  on,  the  enemy  forces  are  assumed 
to  consist  of  aircraft  only;  these  enemy  aircraft  seek  to  enter  the  friend’s  territory;  they  can 
fixe  Air  to  Surface  Missiles  (ASM)  and  Air  to  Air  Missiles  (AAM)  in  order  to  destroy  both 
ground  units  and  airborne  units.  For  their  defense,  the  friends  have  aircraft  that  can  fire 
AAM,  and  ground  units  that  can  fire  Surface  to  Air  Missiles  (SAM)  in  order  to  kill  the 
invading  aircraft.  An  enemy  unit  will  refer  to  aircraft;  a  friendly  asset  will  refer  to  both 
aircraft  and  ground  firing  units;  for  the  neutrals,  a  unit  will  refer  to  an  aircraft  (commercial 
aircraft). 

In  this  model,  the  geometry  has  been  simplified:  it  consists  of  a  straight  line 
(FSCL  or  Fire  Support  Coordination  Line)  separating  the  friendly  forces  from  the  enemy: 
the  system  under  consideration  lies  behind  the  FSCL,  and  hostile  aircraft  are  heading 
towards  this  line  at  speed  V.-  This  model  is  represented  in  Fig.5.2: 


R 


FSCL 


Fig.5.2:  Simplified  IFFN  model 


Rq  is  the  measurement  volume  of  the  system 

Rj  is  the  range  of  the  enemy's  air  surface  missiles 

R  is  the  distance  separating  the  enemy's  aircraft  from  the  FSCL 

V  is  the  speed  of  enemy's  aircraft 

FSCL  is  the  line  between  friends  and  enemies 

The  distance  from  these  aircraft  to  the  FSCL  is  R.  Detecting  units  are  in  charge  of 
the  detection  of  aircraft,  while  engaging  units  are  responsible  for  engaging  them:  an  aircraft 
will  be  detected  by  a  given  detecting  unit  and  engaged  by  another  unit  (engaging  unit):  the 
task  of  the  C^  system  is  to  identify  correctly  an  aircraft  and  to  allocate  it  to  a  given 
engaging  unit 

In  order  to  protect  friendly  assets,  the  enemy's  aircraft  must  be  stopped  before 
they  reach  Rj,  and  can  fur  missiles. 

We  assume  that  the  friendly  aircraft  as  well  as  the  neutral  aircraft  are  flying  at  any 
speed  and  in  any  direction  in  the  diagram  sketched  above. 

In  what  follows,  this  simplified  model  is  developed,  based  on  the  work  carried  out 
by  Logicon  [Logicon  1986];  throughout  the  chapter,  the  notation  defined  in  the  IFFN 
documents  [IFFN  Test  Plan  1985,  Logicon  1986]  will  be  used. 


5.5  PARAMETER  DEFINITION 

To  carry  out  the  effectiveness  analysis  of  this  system,  one  must  first  identify  the 
parameters  (the  independent  variables).  The  parameters  we  are  going  to  consider  are  those 
defined  in  the  IFFN  documentation  [IFFN  Test  Plan  1985,  Logicon  1986];  in  order  to 
design  tests  to  run  on  the  system,  a  test-matrix  has  been  defined;  this  test  matrix  consists  of 
two  sub-matrices:  a  "basic  matrix"  (Fig.5.3)  and  a  "test  excursion  matrix"  (Fig.5.4):  in 
what  follows,  only  one  activity  level  will  be  considered.  The  parameters  are  the  quantities 
that  are  varied  to  go  from  one  element  of  the  test  excursion  matrix  to  an  other.  Each 
element  of  the  basic  matrix  is  called  a  basic  operating  point;  each  basic  operating  point 
associated  with  all  the  possible  test  excursions  is  called  a  test-cell:  therefore,  to  each  basic 
operating  point  is  associated  a  test  cell.  In  the  following  analysis,  we  are  going  to  consider 
one  test  cell  at  a  time,  or  one  element  of  the  basic  matrix  at  a  time:  thus,  only  the  test 
excursion  matrix  is  going  to  be  considered. 


53 


>,A-vv\ 

*  *  **  U  T  MJ 
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Scenario 


Variables 


Activity 
level  1 


Activity 
level  2 


Nominal  performance 
■laximum  system  ID  performance 
Remove  ACP 

Remove  electronic  ID 
Remove  QSA  IFF  nodes 
Remove  JTIDS 

Remove  direct  positive  hostile 
Centralized  mode  of  control 
Remove  NE-3A 
Remove  SIS 


Fig.5.3:  Basic  matrix 


Abreviations: 


JTIDS:  Joint  Tactical  Information  Distribution  System;  this  is  a  high 

capacity  information  distribution  system  providing  secure, 
flexible  and  jam  resistant  information  transfer  among  the  nodes 
of  the  system. 

NE-3A:  NATO  airborne  early  warning  sytem  (Boeing  707);  this  is  a 

high  altitude  detection  aircraft. 

SIS:  Special  Information  System;  this  is  a  source  of  intelligence 

information  available  to  Control  and  Reporting  Center  (CRC) 
nodes. 

ACP:  Air  Control  Procedure:  this  is  the  set  of  rules  and  protocols  that 

enable  the  system  to  monitor  efficiently  the  airspace. 

Q&A  IFF  nodes:  Question  and  Answer  devices  for  Identification  Friend  Foe; 

these  are  devices  located  at  the  weapon  level  that  provide  local 
ID  information  for  a  given  weapon. 


Scenario 


Activity 


Activity 


Variables 


level  1 


level  2 


SIS  into  CRC 


Variant  ACP 


Perfect  10 


Decentralized  mode  of  control 


Improved  QtA  IFF  devices 


Fig.5.4:  Test  Excursion  matrix 


For  the  sake  of  the  analysis  to  come,  the  parameters  will  be: 

-  time  needed  to  pass  information  between  two  nodes  (P}) 

-  range  from  aircraft  to  FSCL  at  time  of  detection  (P2) 

-  quality  of  identification  (P3) 

-  level  of  centralization  of  control  (P4) 

-  quality  of  target  allocation  and  engagement  (P5) 


These  parameters  correspond  to  the  test  excursion  matrix;  they  are  independent 
variables  included  in  [0,1]: 

Pj  corresponds  to  the  SIS  (Special  Information  System)  included  or  not  into  the 
CRC  (Control  and  Reporting  Center);  this  fact  is  modeled  by  a  varying  the  time  delay 
required  to  pass  information  between  two  nodes. 

P2  corresponds  to  the  variable  ACP  (Air  Control  Procedure):  the  ACP  is  the  set  of 
rules  that  enables  the  system  to  monitor  the  air  space;  the  effect  of  varying  the  ACP  is 
assumed  to  be  the  variation  of  the  range  from  aircraft  to  FSCL  at  time  of  detection:  the 
better  the  ACP,  the  larger  this  range. 

P3  corresponds  to  the  variation  in  the  quality  of  identification  (ID). 

P4  corresponds  to  the  level  of  centralization. 

P5  corresponds  to  the  quality  of  the  Q&A  IFF  devices  (Question  and  Answer 
devices  for  Identification  Friend  Foe):  since  Q&A  IFF  devices  provide  local  ID 
information  at  the  weapon  level,  the  quality  of  these  components  has  a  direct  effect  on 
allocation  and  engagement  performances. 


These  parameters  reflect  the  experiments  that  will  be  conducted  on  the  IFFN 
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testbed:  basic  operating  points  are  represented  by  the  basic- matrix;  from  these  basic 
operating  points,  excursions  wilJ  be  considered.  These  excursion  will  be  conducted  by 
varying  the  parameters  defined  above.  To  each  basic  operating  point  corresponds  a  "test 
cell".  Only  the  five  parameters  defined  above  will  be  varied  to  apply  the  SEA  methodology: 
other  parameters  describing  either  the  system  or  the  context  will  be  fixed  in  the  SEA 
analysis. 

5.6  MEASURES  OF  PERFORMANCE 

5.6.1  Introduction 

After  having  defined  the  parameters,  one  must  specify  the  MOPs  of  interest  for  the 
system  at  hand.  These  MOPs  must  allow  one  to  make  a  decision  concerning  the  system: 
they  must  have  a  clear  physical  interpretation. 

5.6.2  Notation 
Let  us  denote: 

x(t):  number  of  friends  at  time  t 

xq:  initial  number  of  friends 

y(t):  number  of  enemies  at  time  t 

yg:  initial  number  of  enemies 

z(t):  number  of  neutrals  at  time  t 

zq:  initial  number  of  neutrals 

n(t):  fraction  of  friendly  forces  lost  at  time  t 

m(t):  fraction  of  enemy  forces  at  time  t 

n(t)=  (xq-  x(t))  /xg  ,  m(t)=  (yQ- y(t))/y0  (5.1) 

Quantities  xg.yg.zg  are  measured  at  the  initial  time:  that  is  the  time  when  hostile  aircraft 
enter  the  detection  volume  (Rg)  of  the  system. 

Quantities  x(Tf),y(Tf),z(Tf)  are  measured  at  the  final  time  Tf.  The  battle  stops  when  either 
the  friendly  forces  or  the  enemy's  have  lost  a  given  fraction  of  their  assets.  The  final  time 
Tf  is  defined  by 


(5.2) 


( n(t)  <  nf  and  m(t)  <  mf  for  all  0  <  t  <  Tf ), 
and  ( n(Tf)  =  nf  or  m(Tf)  =  mf ) 

where  nf  and  mf  represent  the  strategy  of  each  side:  since  the  friendly  forces  are  defending 
their  own  territory,  they  are  probably  willing  to  loose  a  greater  fraction  of  their  forces  than 
the  enemy:  nf>mf  is  very  likely  to  be  true. 

5.6.3  MOP  Definition 

In  order  to  enable  one  to  evaluate  the  system,  the  MOPs  we  will  consider  must 
have  a  clear  physical  meaning:  as  pointed  out  in  section  5.3,  we  want  the  system  to 
perform  a  threefold  task:  deter  enemy  from  entering  the  friend’s  territory,  stop  the  enemy 
as  far  as  possible  from  this  territory  and  before  it  can  fire  missiles  aimed  at  friendly  assets, 
and  kill  as  few  neutral  as  possible. 

To  evaluate  the  first  task,  we  need  a  quantity  indicating  whether  the  friends  win 
the  battle  or  not;  an  indicator  of  the  willingness  of  the  friends  to  keep  on  fighting  is  the 
ratio 

x(Tf)/(xo*(l-nf))  =  (l-n)/(l-nf).  (5.3) 

This  ratio  measures  how  far  the  remaining  forces  of  the  friends  are  from  their  lowest 
acceptale  level  as  given  by  1-nf.  If  the  friends  win  the  battle,  this  ratio  will  be  greater  than 
one;  if  they  loose,  it  will  be  equal  to  one  since  they  are  giving  up  when  their  level  of  losses 
reaches  nf.  A  similar  ratio  can  be  defined  for  the  enemy.  Then,  we  will  consider  as  the  first 
MOP  of  our  problem,  the  ratio  of  these  two  ratios:  this  ratio  of  ratios  will  compare  the 
willingness  of  the  two  opponents  to  keep  on  fighting.  Thus  we  define 

MOP1  =  [x(Tf)/(x0*(l-nf))]/Iy(Tf)/(y0*(l-mf))]  (5.4) 

The  second  quantity  we  want  to  evaluate  is  the  number  of  neutral  aircraft  killed  by 
the  friends;  indeed,  since  we  are  interested  in  evaluating  the  friend's  system  as  opposed  to 
he  enemy's  one,  we  consider  only  the  neutrals  shot  down  by  the  friend’s  air-defense.  For 
simplicity,  we  assume  that  the  enemy  is  not  killing  any  neutral,  and  that,  if  a  neutral  is 
killed,  it  is  by  a  friend.  Then,  we  are  interested  in  the  number  of  neutrals  remaining  at  the 
end  of  the  battle;  the  MOP  that  will  measure  this  performance  of  the  system  is: 
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M0P2  =  z(Tf)/z Q 


(5.5) 


The  last  quantity  of  interest  is  the  distance  of  the  enemy  from  the  FSCL  when  the 
battle  ends.  This  is  measured  by  the  following  ratio: 

MOP3  =  (Distance  of  the  remaining  enemy's  aircraft  at  time  Tf)/Rj 

5.6.4  Interpretation  of  the  MOPs: 

If  MOPl>l,  the  hostile  aircraft  fly  back  because  their  level  of  losses  has  reached 


mf  while  n(Tf)<nf ;  indeed,  if  M0P1>1,  then 

[x(Tf)/(xo*(l-nf))]>[y(Tf)/(yo*(l-mf))]  (5.6) 

and  one  of  the  two  terminating  condition  is  true 

l-y(Tf)/Y0  =  mf  or  l-x(Tf)/xQ  =  nf  .  (5.7) 

Let  us  suppose  that  the  latter  of  these  two  equalities  is  true.  Then 

y(Tf)/(y0*(l-mf))  <  1  (5.8) 

which  is  equivalent  to 

(yo-yCTfWyo  >  mf  •  (5-9) 

Therefore,  if 

MOP1  >  1  and  1-nf  =  x(Tf)/xQ  , 


then  the  level  of  loss  for  the  enemy  at  Tf  is  greater  that  its  maximum:  it  is  impossible.  The 
conclusion  is  that,  if 


MOP1  >  1 ,  then  1-mf  =  y(Tf)/YQ 


(5.10) 


and  the  attack  has  been  repulsed. 

If  M0P1<1,  the  friendly  forces  give  up  and  loose  the  battle  because  their  level  of 
losses  has  reached  nf  while  m(Tf)<m^:  the  proof  is  the  same  as  the  one  above. 

MOP2  has  a  straightforward  interpretation:  since  we  assume  that  only  the  friends 
arc  killing  neutral  assets,  MOP2  is  the  fraction  of  neutral  aircraft  remaining  at  the  end  of  the 
battle. 

MOP3  represents  a  ratio  of  distance  to  the  FSCL 

If  M0P3>1,  the  enemy  aircraft  are  stopped  before  they  can  fire  missiles  aimed  at 
friendly  assets. 

If  MOP3<l,  the  enemy  aircraft  can  fire  missiles  before  being  stopped.  The  greater 
this  ratio  is,  the  better  it  is  for  the  friendly  forces. 

5.7  MAPPING  FROM  THE  PARAMETER  SPACE  INTO  THE  MOP  SPACE 
5.7.1  Introduction 

In  this  part  we  are  going  to  map  the  parameter  space  into  the  MOP  space;  the  basis 
of  this  section  is  the  work  completed  by  Logicon  [Logicon  1986],  where  about  thirty 
quantities  representing  the  way  the  IFFN  system  performs  are  defined.  Most  of  these 
quantities  are  conditional  probabilities  that  describe  the  different  stage  of  the  air-defense 
process.  These  quantities  are  called  MOPs  and  MOEs  in  the  Logicon  documentation;  since 
they  are  different  from  the  MOPs  and  MOEs  we  consider  for  the  SEA  methodology,  and 
since  we  want  to  keep  the  notation  defined  by  Logicon,  these  quantities  will  be  denoted 
"Mop"  and  "Moe"  as  opposed  to  "MOP"  and  "MOE"  in  the  SEA  methodology. 

First  of  all,  we  are  going  to  aggregate  the  conditional  probabilities  (Mops  and 
Moes)  defined  in  the  IFFN  documents  [Logicon  1986]  in  a  smaller  number  of  quantities; 
then,  we  will  determine  the  values  of  these  aggregate  quantities  in  terms  of  the  independent 
variables  of  the  problem  (parameters);  finally,  we  will  use  a  Lanchester  model  [Echkian 
1982,  Taylor  1974,  Moose  and  Wozencraft  1983]  to  derive  the  MOPs  (for  our  problem) 
in  terms  of  the  parameters. 
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5.7.2  Aggregation  of  the  IFFN  Mops  and  Moes 

The  IFFN  documents  define  conditional  probabilities  (Mops  and  Moes)  that  are 
useful  for  setting  up  a  mathematical  model  of  the  system.  These  quantities  correspond  to 
stages  in  the  ID  process;  these  stages  have  been  described  in  the  IFFN  documentation 
[Logicon,  1986],  and  in  Gandee  (1986).  The  purpose  of  this  section  is  to  aggregate  these 
stages  in  order  to  determine  four  basic  quantities:  probability  of  engaging,  a  friend,  a 
neutral,  or  an  hostile,  and  the  time  elapsed  between  detection  and  engagement. 

The  basic  stages  of  the  model  are:  detection,  identification  (ID),  comparison 
between  different  IDs  coming  from  different  detecting  units,  conflict  resolution,  allocation, 
and  engagement.  A  conditional  probability  is  associated  to  each  of  these  stages.  To  these 
six  stages  corresponding  to  physical  processes,  one  must  add  a  fictitious  step  that 
describes  the  probability  of  true  identification.  With  a  decision  tree  [White  1985],  it  is  then 
possible  to  determine  three  basic  quantities:  probability  to  engage  a  friend,  a  neutral,  or  an 
enemy. 

The  decision  trees  are  presented  in  figures  5.5,  5.6,  and  5.7;  each  of  these  trees  is 
conditioned  on  the  true  ID  of  an  aircraft:  for  example,  on  the  first  tree  (Fig.5.5),  all  the 
probabilities  assume  that  the  true  ID  of  the  aircraft  under  consideration  is  "friend".  Let  us 
now  review  the  stages  of  these  trees  with  the  notation  defined  in  the  IFFN  documents 
[IFFN  TestPlan  1985,  Logicon  1986]: 

-"Detection":  probability  of  detecting  an  aircraft  (Moel) 

-"ID":  probability  of  identifying  (ID)  an  aircraft  (Moe3) 

-"ID  =  True":  this  is  a  fictitious  physical  process  that  allows  one  to  take  into  account 
the  possibility  of  wrong  ID  in  the  decision  trees.  Because  of  the  three  different 
types  of  aircraft,  there  are  three  possible  outcomes  for  this  stage:  ID  as  a  friend, 
a  neutral  or  a  hostile.  In  the  decision  trees  appear  the  quantities  P(ix/yd): 
probablity  of  identifying  an  aircraft  as  x,  given  its  true  identity  is  y  and  it  has 
been  detected,  where  x  and  y  can  be  "friend"  (0,  "neutral"  (n),  or  "hostile"  (h) 
-"ID  passed  =  ID  recorded":  represents  the  comparison  between  IDs  coming  from 
different  detecting  units:  Mop3.5  is  the  probability  that  there  is  no  conflict. 
-"Conflict  resolution":  in  case  of  conflict  between  different  IDs,  this  stage  represents 
the  probability  of  resolving  this  conflict  (Mop3.6). 

-"Allocate":  represents  the  allocation  process.  Mop4.1  has  to  be  broken  up  into  three 
different  probabilities  for  the  three  trees  considered: 

P(a/xi):  probability  of  allocating  an  aircraft,  given  its  true  ID  is  x  and  it  has  been 
identified,  where  x  can  be  "friend"  (f),  "neutral"  (n),  or  "hostile  (h). 
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Prob(Y«*0  ID  P***«d-IO  r«corde 


Decision  Tree  for  Friends 


Decision  Tree  for  Neutrals 


Decision  Tree  for  Hostiles 


-"Engage":  represents  the  engagement  process.  Mop5.1  has  to  be  broken  up  into 
three  different  probabilities  for  the  three  trees  considered: 

P(e/xi):  probability  of  engaging  an  aircraft,  given  its  true  ID  is  x  and  it  has  been 
identified,  where  x  can  be  "friend"  (f),  "neutral"  (n),  or  "hostile  (h). 

Some  simplifications  have  been  made  in  the  work  completed  by  Logicon  [Logicon 

1986]: 

-since  we  are  interested  only  in  the  indirect  ID  problem,  we  do  not  take  into 
account  the  'local  threat  evaluation"  stage  defined  in  the  IFFN  documentation. 

-since  the  tracking  stage  defined  in  Logicon  [1986]  doesn't  affect  the  probabilities 
considered  in  our  model,  this  stage  has  been  removed  from  the  analysis. 

-only  a  two  stages  verification  process  is  considered,  instead  of  a  multi-stage  one. 
-the  stages  "ID  passed"  and  "ID  passed  =  ID  recorded"  defined  in  Logicon  [1986] 
have  been  aggregated  in  a  single  stage  ("ID  passed  =  ID  recorded"). 

These  decision  trees  enable  one  to  determine  three  basic  quantities:  probability  to 
engage  a  friend  (Moe7),  a  neutral  (Moe8),  or  a  hostile  (Moe9).  Since 


P(ih/fd)  +  P(in/fd)  +  P(if/fd)  =  1  ,  (5.11) 

P(ih/nd)  +  P(in/nd)  +  P(if/nd)  =  1  ,  (5. 12) 

P(ih/hd)  +  P(in/hd)  +  P(ifhfd)  =  1  ,  (5.13) 

one  can  deduce: 

Moe7  *  Moel*Moe3*P(ih/fd)*P(a/fi)*P(e/fa)*A  (5. 14) 


with  A  =  (l-P(ih/fd))*(l-Mop3.5)*Mop3.6  +  Mop3.5 
+  P(ih/fd)*(l-MOP3.5)*Mop3.6  ; 

Moe8  =  Moel*Moe3*P(ih/nd)*P(a/ni)*P(e/na)*B  (5.15) 

with  B  =  (l-P(ih/nd))*(l-Mop3.5)*Mop3.6  +  Mop3.5 
+  P(ih/nd)*(l-MOP3.5)*Mop3.6 


Moe9  =  Moei*Moe3*P(ih/hd)*P(a/hi)*P(e/ha)*C 


(5.16) 


with  C  =  (l-P(ih/hd))*(l-Mop3.5)*Mop3.6  +  Mop3.5 
+  P(ih/hd)  *(  1  -Mop3.5)*Mop3.6 


It  is  also  possible  with  these  decision  trees  to  determine  the  "mean  time  elapsed 
between  detection  and  engagement"  (MoelO). 

MoelO  =  Mop2.3  +  Mop3.4  +■  Mop4.4  +  Mop5.3 

where  (Mop2.3+Mop3.4)  is  the  time  between  detection  and  ID,  Mop5.3  is  the  time  elapsed 
between  allocation  and  engagement,  and  Mop4.4  is  the  time  elapsed  between  ID  and 
allocation  [Logicon  1986]:  only  this  latter  time  is  assumed  to  vary,  depending  on  whether 
there  is  a  conflict  during  the  ID  process  or  not  Therefore,  with  these  assumptions, 

Mop2.3  +  Mop3.4  +  Mop5.3  *  constant,  (5. 17) 

Mop4.4  *  Mop3.7+(l-Mop3.5)*Mop3.7  (5.18) 

and 

MoelO  »  Mop2.3  +  Mop3.4  +  Mop4.4  +  Mop5.3  .  (5. 19) 

5.7.3  From  the  parameters  to  the  aggregate  quantities 

In  this  section  we  are  going  to  express,  the  Mops  and  Moes  considered  in  the 
previous  section  in  terms  of  the  five  parameters. 

Mopl.2  (distance  from  aircraft  to  FSCL  at  time  of  detection)  is  assumed  to  be 
affected  only  by  the  variation  of  the  Air  Control  Procedure  (ACP):  the  ACP  is  the  set  of 
rules  that  enable  the  system  to  monitor  the  air-space.  The  better  the  ACP,  the  greater 
Mopl.2  is:  indeed,  with  a  good  ACP,  one  is  able  to  detect  an  enemy  within  a  very  short 
period  of  time;  it  yields  a  large  Mopl.2  because  within  this  time  period,  the  aircraft  cannot 
go  very  far  inside  the  detection  volune.  We  assume: 


Mopl.2  =  Rq*P2  0<P2<1  (5.20) 

The  probabilities  corresponding  to  MOP3.1  are  assumed  to  be  only  functions  of 
P3.  If  P3  equal  to  1  corresponds  to  perfect  ID  capabilities,  we  have  the  following  relations: 

P(ih/hd)  =  P3  ,  (5.21) 

P(ih/nd)  =  P(ih/fd)  =  1-P3  ,  0<  P3  <1  (5.22) 


The  probabilities  Mop3.5  (ID  passed  =  ID  recorded)  and  Mop3.6  (conflict 
resolution)  are  assumed  to  be  only  function  of  the  level  of  centralization.  The  probability  of 
conflict  (Mop3.5)  must  decrease,  and  the  probability  of  conflict  resolution  (Mop3.6)  must 
increase  when  the  process  becomes  more  centralized  (P4=l  when  centralization  is  total). 

Mop 3. 5  =  0.75  -  0.5*P4  (5.23) 

Mop3.6  =  0.25  +  0.5*P4  0<P4<1  .  (5.24) 

The  time  needed  to  exchange  ID  information  between  a  unit  that  produces  this 
information  and  another  that  uses  it  (Mop3.7)  is  assumed  to  be  a  function  of  the  level  of 
centralization  (P^  and  of  the  integration  of  the  Special  Information  System  (SIS)  into  the 
Control  and  Reporting  Center  (CRC)  (Pj).  If  the  SIS  is  included  into  the  CRC  (Pj=0), 
Mop3.7  decreases;  if  the  process  is  centralized,  Mop3.7  increases:  these  assumptions  are 
justified  by  the  two  following  remarks:  to  exchange  ID  information  between  two  given 
components,  the  greater  the  level  of  centralization  (the  greater  P^,  the  greater  the  number 
of  information  exchanges  between  nodes  is,  and  the  greater  the  time  is;  if  the  SIS  is 
included  into  the  CRC  (Pj  decreases),  an  important  database  is  included  into  a  node  of  the 
system,  thus  reducing  the  overall  time  required  for  exchanging  ID  information.  If  we 
assume  that  Pj  and  P4  are  weighted  evenly,  and  that  the  time  to  exchange  information 
cannot  be  smaller  than  (Mop3.7)max/2,  we  have  the  following  relation: 

Mop3.7  =  (Mop3.7)max*(0.5  +  0.25*?!  +0.25*P4) ,  (5.25) 

where  (Mop3.7)max  is  a  constant  and  0  <  PI  <  1  . 

The  probabilities  of  allocation  and  engagement  are  assumed  to  be  only  functions  of 
P5.  With  improved  Question  and  Answer  devices  for  Identidication  Friend  Foe  (Q&A  IFF 
devices),  that  is  if  P5  increases,  P(a/hi)  and  P(e/ha)  are-going  to  increase,  while  P(a/fi), 
P(e/fa),  P(a/ni),  and  P(e/na)  are  going  to  decrease.  This  is  justified  by  the  fact  that  Q&A 
IFF  devices  are  components  that  are  at  the  weapon  level:  if  the  true  ID  of  an  aircraft  is 
friend  or  neutral,  the  better  the  quality  of  these  devices,  the  lower  the  probability  this 
aircraft  will  be  allocated  and  engaged  is;  if  the  true  ID  of  the  aircraft  is  hostile,  the  greater 
the  probability  of  allocating  it  and  engaging  it  is.  We  assume  the  following  relations: 

1/2 


PCa/hi)  =  P(e/ha)  =  (0.875  +  0. 125*P5) 


(5.26) 


P(a/fi)  =  P(c/fa)  *  P(a/ni)  =  P(c/na) 

=  (0.125  -0.125*P5)1/2  ,  (5.27) 

with  0  <  P5  <  1 . 

In  the  equations  of  the  model  only  the  probabilities  of  "allocating  and  engaging"  an 
aircraft  are  considered:  only  the  products  P(a/xi)*P(e/xa),  where  x  is  the  true  ID  of  the 
aircraft,  are  considered;  in  order  to  keep  these  products  as  linear  functions  of  P5,  an 
exponent  1/2  has  been  introduced  in  expressions  5.26  and  5.27. 

We  can  now  express  the  four  aggregate  quantities  we  are  interested  in  (Moe7, 
Moe8,  Moe9,  MoelO)  in  terms  of  the  parameters  of  the  problem.  Some  constants  remain  to 
be  fixed  (Moel,  Moe3,  Mop3.7max,  Rq  ...);  since  these  constants  are  not  going  to  be 
varied  the  System  Effectiveness  Analysis,  they  are  assumed  to  be  given  quantities  for  each 
basic  operating  point  to  which  the  methodology  is  applied. 

In  the  next  section,  we  will  determine  the  MOPs  of  our  problem  in  term  of  these 
four  quantities. 

5.7.4  Lanchester  model 

The  final  step  to  determine  the  three  MOPs  we  are  going  to  consider  is  the  use  of 
the  Lanchester  equations  [Ekchian  1982,  Taylor  1974,  Moose  and  Wozencraft  1983]:  from 
an  initial  number  of  aircraft  of  each  type,  we  determine  the  final  number  in  each  category 
(x(Tf),y(Tf),z(Tf))  on  the  basis  of  the  probabilities  computed  with  the  decision  trees;  the 
final  time  is  a  crucial  issue  because  the  Lanchester  models  are  valid  only  for  a  great  number 
of  entires:  they  are  aggregate  models.  The  Lanchester  equations  provide  a  means  to 
compute  the  number  of  friends,  foes,  and  neutrals  at  each  time  of  the  battle;  with  the 
variation  of  the  number  of  friends  and  hostiles  during  the  battle,  one  can  determine  the 
terminating  time  Tf  using  the  criteria  of  equation  5.2;  this  terminating  time  enables  one  to 
obtain  the  number  of  friends  (x(Tf)),  foes  (y(Tf)),  and  neutrals  (z(Tf))  at  the  end  of  the 
battle.  These  are  the  three  quantities  that  one  needs  to  evaluate  MOP1  and  MOP2  (see 
equations  5.4  and  5.5).  The  third  MOP  of  our  model  is  then  equal  to 

MOP3  =  (Mop  1 .2- V*Tf)/R  j  (5.28) 

(Mop  1.2  is  the  range  from  aircraft  to  FSCL  at  time  of  detection). 


In  the  Lanchester  model  we  are  going  to  consider,  we  assume  that  all  the  friends 
are  within  the  weapon  range  of  the  enemy,  and  that  all  the  neutrals  and  all  the  hostiles  are 
within  the  weapon  range  of  the  friend's  units.  Since  we  are  interested  in  the  performance 
of  the  indirect  ID  process,  we  consider  losses  in  friendly  forces  to  be  due  to  the  enemy 
action,  and  to  errors  within  the  friend  ID  process;  we  consider  losses  in  the  enemy  forces, 
and  neutral  losses  to  be  due  to  the  friend's  fire  only:  indeed,  we  are  interested  in  the 
performance  of  the  friend's  air  defense  system  only. 

For  the  firing  relation  between  the  friends  and  the  enemy,  we  assume  that  each 
unit  is  aware  of  the  exact  location  of  the  remaining  enemy  units,  so  that  when  a  target  is 
destroyed,  fire  may  be  immediately  shifted  to  a  new  target:  it  means  that,  if  we  consider 
only  x  (Friend)  and  y  (Enemy)  firing  at  each  other  without  any  error,  we  have: 

[dx/dt]/y  and  [dy/dt]/x 

be  constant  over  time  (x  losses  are  only  dependent  on  the  level  of  forces  y,  because  each  y 
unit  knows  exactly  where  to  fire  at  an  x  unit).  Thus,  for  the  interaction  between  x  and  y, 
we  have: 

dx/dt  =  -b*y  , 
and  dy/dt  =  -c*x  , 

where  b  and  c  are  positive  constants  (Lanchester  square  law).  Since  we  want  to  consider 
errors  in  the  ID  process  for  the  friends,  we  must  add  a  new  term  in  the  friend  losses:  this 
term  is  proportional  to  x  and  the  evolution  of  x  is  then  described  by: 

dx/dt  -  -a*x  -b*y  . 

We  assume  that  the  neutral  losses  are  only  due  to  the  friend's  errors,  and  that  the 
friend's  fire  on  the  neutrals  is  uniformly  distributed  over  a  constant  target  area:  therefore, 
we  are  going  to  use  a  linear  attrition  law: 

dz/dt  =  -d*x*z , 


where  d  is  a  positive  constant.  This  law  means  that  the  rate  of  destruction  of  neutral  aircraft 
produced  by  each  unit  x  depends  linearly  on  z:  it  is  consistent  with  the  expectation  that,  as 
the  number  of  neutral  aircraft  decreases,  their  rate  of  destruction  decreases  also  t  there  will 
be  less  noise  in  the  ID  process  and  therefore  less  error). 

Therefore,  we  have  the  following  three  equations: 
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dx/dt  =  -a*x  -b*y 

x=x()  at  t=<) 

(5.29) 

dy/dt  =  -c*x 

_v=vq  at  t=0 

(5.30) 

dz/dt  =  -d*x*z 

z=Zg  at  t=<) 

(5.31) 

for  0  <  t  <  Tf,  Tf  determined  as  shown  above  (equation  5.2). 

"a”  is  the  probability  of  engaging  a  friend  per  unit  of  time:  thus, 
a  =  Moe7/Moel0  .  (5.32) 

Similarly, 

c  =  Moe9/Moel0  (5.33) 

and,  d  =  Moe8/MoelO  (5.34) 

b  is  assumed  to  be  exogenous  and  fixed  independently  of  the  parameters:  indeed,  b  is  the 
probability  for  an  enemy  to  kill  a  friend;  it  is  an  exogenous  quantity. 

The  integration  of  equations  5.29,  5.30,  5.31  yields: 


x  =  B|*exp(-(a+s)*t/2]  +  B2*exp(-(a-s)*t/2]  (5.34) 

y  =  B3*exp[-(a+s)*t/2]  +  B4*exp[-(a-s)*t/2]  (5.35) 

z  =  zo*exp[d*(y-y0)/c]  (5.36) 

with,  s  =[a*a  +  4*b*c]  ^  ,  (5.37) 

1*1  =  (a+s)*[xQ  -(a-s)*yo/(2*c)]/(2*s)  (5.38) 

-  (a-s)*[(a+s)*yo/(2*c)  -  xg]/(2*s)  (5.39) 

B3  =  c*(x0  -(a-s)*yo/(2*c)]/s  (5.40) 

B4  =  c*[(a+s)*yo/(2*c)  -  x0]/s  (5.41) 

The  final  time  Tf  is  given  by 

n(Tf)  =  nf  (5.42) 

or,  m(Tf)=mf  (5.43) 


Since  one  cannot  find  a  closed  expression  for  Tf,  the  final  time  must  be  computed 
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numerically. 


fcv 


We  can  now  compute  the  three  MOPs  (MOP1,  MOP2,  MOP3)  of  the  system  for 
each  point  (Pj,  P2,  P3,  P4,  P5)  in  the  parameter  space: 

MOP1  =  [x(Tf)/(x0*(l-nf))y[y(Tf)/(y0*(l-mf))]  (5.4) 

MOP2  =  z(Tf)/z0  (5.5) 

MOP3  =  (Mop  1.2  -  V^TfJ/R!  (5.28) 


5.8  EVOLUTION  OF  THE  SYSTEM 

5.8.1  Introduction 

Up  to  this  point,  we  have  assumed  the  system  to  be  at  a  basic  operating  point  in 
the  basic  test  matrix:  for  each  basic  operating  point,  a  simplified  model  has  been  presented 
in  order  to  apply  the  System  Effectiveness  Analysis  methodology. 

In  this  section,  we  define  five  basic  operating  points  by  assigning  different  values 
to  the  constants  that  remain  to  be  fixed  (Moel,  Moe3,  Mop3.7max  ...).  Thus,  we  will 
have  five  basic  operating  points  corresponding  to  four  different  sets  of  constants  in  the 
model.  For  each  of  these  basic  operating  point,  the  excursion  matrix  will  be  explored  by 
varying  the  parameters  P^,  P2,  P3,  P4,  P5. 

5.8.2  Nominal  performance 

This  is  the  basic  operating  point  that  corresponds  to  the  nominal  level  of 
performance  in  the  system;  as  defined  in  the  IFFN  documents  [IFFN  Test  Plan,  1985],  the 
nominal  system  includes  all  the  components  that  can  be  removed  to  go  from  one  test  cell  to 
another  (Fig.5.3):  in  particular  it  includes  the  NATO  airborne  early  warning  (NE-3A),  and 
the  Joint  Tactical  Information  distribution  System  (JTIDS).  As  mentioned  in  section  5.3, 
an  unit  refers  to  either  an  aircraft  or  a  ground  firing  unit  for  the  friend;  it  refers  to  an  aircraft 
for  the  enemy  as  well  as  for  the  neutrals.  We  assume  the  following  values  for  the  constants 
of  the  model: 

V  =  200  meters  per  second 

xq  =  100  units 
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yo  =  150  units 
zq  =  50  units 
b  *  0.022  units  per  second 
nf  =  0.6 

mf  =  0.3 

Rq  *  10000  meters 
Rj  *  5000  meters 

Mop2.3  +  Mop3.4  +  Mop  5.3  *  7  seconds 
(Mop3.7)max  =  10  seconds 
Moel  =  Moe3  =  1 

5.8.3  Removal  of  the  NE-3A 

This  basic  operating  point  corresponds  to  the  system  defined  in  section  5.8.2 
without  the  NATO  airborne  early  warning  system  (NE-3A).  We  assume  that  the  constants 
are  the  same  as  above  except  for  the  following  ones: 

Rq  *  9000  meters 
Moel  »  Moe3  =  0.98 

The  removal  of  the  NE-3A  represents  the  removal  of  a  source  of  information; 
without  NE-3A,  the  system  cannot  monitor  as  large  a  detection  volume  as  the  nominal 
system;  the  removal  of  the  NE-3A  corresponds  to  a  reduction  in  the  detection  and  ID 
capabilities  of  the  system,  and  therefore  to  a  deterioration  in  the  detection  and  ID 
performances. 

5.8.4  Removal  of  the  JTIDS 

This  basic  operating  point  corresponds  to  the  system  defined  in  section  5.8.2 
where  the  Joint  Tactical  Information  Distribution  System  (JTIDS)  is  not  present.  We 
assume  that  the  constants  are  the  same  as  in  section  5.8.2  except  for  the  following  ones: 

Mop2.3  +  Mop3.4  +  Mop  5.3  =  8  seconds 
(Mop3.7)max  =  12  seconds 
Moel  =  Moe3  =  0.98 


The  JTIDS  is  a  high  capacity  information  distribution  system  that  provides  secure, 
flexible  and  jam  resistant  information  transfer  among  all  the  nodes  of  the  system;  its 
removal  corresponds  to  increases  in  communication  delays,  and  to  less  reliable  detection 
and  ID  capabilities. 

5.8.5  Removal  of  the  JT1DS  and  of  the  NE-3A 

This  basic  operating  point  corresponds  to  the  system  defined  in  section  5.8.2 
where  the  JTIDS  and  the  NE-3A  have  been  removed.  We  assume  that  the  constants  are  the 
same  as  in  section  5.8.2  except  for  the  following  ones: 

Rq  =  9000  meters 

Mop2.3  +  Mop3.4  +  Mop  5.3  =  8  seconds 
(Mop3.7)mflT  =  12  seconds 
Moel  =  Moe3  =  0.95 

These  values  are  a  combination  of  the  ones  assumed  in  sub-sections  5.8.3  and 

5.8.4. 

5.8.6  Removal  of  the  JTIDS  and  inclusion  of  an  additional  NE-3A 

This  basic  operating  point  corresponds  to  the  nominal  system  where  the  JTIDS 
has  been  removed  and  where  an  additional  NE-3A  has  been  included  in  the  system.  This 
point  is  not  defined  in  the  IFFN  documents  [IFFN  Test  Plan  1985]  and  does  not 
correspond  to  any  point  in  the  basic  matrix  of  Fig.5.3;  nevertheless  this  configuration  will 
be  considered  later  to  apply  the  dynamic  programming  algorithm  presented  in  Chapter  4. 
we  assume  that  the  constant  are  the  same  as  in  section  5.8.2  except  for  the  following  ones: 

Mop2.3  +  Mop3.4  +  Mop  5.3  =  8  seconds 
(Mop3.7)max  =  12  seconds 
Moel  =  Moe3  =  1 

The  inclusion  of  an  additional  NE-3A  into  the  system  does  not  extend  the  detection 
volume.  If  one  compares  this  configuration  to  the  one  defined  in  section  5.8.4  (nominal 
system  without  JTIDS),  one  should  note  that  the  inclusion  of  an  additional  NE-3A  in  the 
system  only  affects  the  probability  of  detection  (Moel)  and  the  probability  of  identification 
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(Moc3):  these  two  quantities  are  assumed  to  increase  as  an  additional  NE-3A  gets  included 
in  the  system. 

In  this  section  we  have  defined  five  basic  operating  points  to  which  the  SEA 
methodology  will  be  applied. 

5.9  CONCLUSION 

In  this  chapter,  an  actual  air  defense  system  has  been  presented,  and  a  simplified 
mathematical  model  of  this  system  has  been  set  up;  this  mathematical  model  will  be  the 
basis  for  the  design  of  experiments  to  run  on  the  test-bed.  To  this  model  (with  a  5 
dimensional  parameter  space  and  a  3  dimensional  MOP  space),  we  will  apply  the 
experiment  design  algorithm  described  in  chapter  3;  this  algorithm  will  enable  us  to 
determine  a  small  number  of  experiments  to  run  on  the  test-bed,  and  then  to  reconstitute  the 
system  locus  of  the  actual  system/test-bed.  To  the  actual  system  we  will  then  apply  the 
dynamic  programming  algorithm  introduced  in  chapter  4. 

The  purpose  of  this  example  is  to  demonstrate  that  the  methodologies  introduced 
in  previous  chapters  can  be  useful  in  solving  a  real  engineering  problem:  evaluate 
experimentaly  a  system  and  improve  it. 

In  chapter  6  we  will  present  the  results  of  the  System  Effectiveness  Analysis 
applied  to  the  DFFN  simplified  mathematical  model  described  in  this  chapter. 

Finaly,  in  chapter  7,  the  demonstration  of  the  procedures  introduced  in  chapter  3 
and  4  will  be  carried  out 


CHAPTER  6 

RESULTS  OF  THE  SYSTEM  EFFECTIVENESS  ANALYSIS 
METHODOLOGY  APLLIED  TO  THE  SIMPLIFIED  IFFN  MODEL 


6.1  INTRODUCTION 

The  results  of  the  System  Effectiveness  Analysis  methodology  applied  to  the 
mathematical  model  developed  in  chapter  5  are  presented  in  this  chapter.  The  SEA 
methodology  will  be  applied  to  each  of  the  five  basic  operating  points  defined  in  the 
previous  chapter  by  varying  parameters  Pj  through  P5. 

We  will  focus  primarily  on  the  basic  operating  point  corresponding  to  the  nominal 
level  of  performance  of  the  IFFN  system.  Since  the  results  are  qualitatively  the  same  for 
the  four  other  operating  points,  they  will  be  presented  in  less  detail. 

Plots  are  presented  throughout  this  chapter  the  data  have  been  generated  by 
solving  the  equations  of  the  IFFN  problem  on  a  personal  computer  (IBM  PC/AT).  The 
graphics  have  been  generated  with  a  package  [Bohner  1986]  developed  on  the  same 
machine,  and  plotted  using  an  HP  7475 A  plotter. 


6.2  MISSION  LOCUS 

6.2.1  Introduction 

We  assume  that  the  mission  the  system  has  to  perform  is  the  same  for  the  four 
basic  operating  points.  Therefore,  we  only  need  to  define  one  mission  locus  for  the 
system. 

6.2.2  Mission  Requirements 

As  mentioned  in  chapter  2,  in  order  to  enable  one  to  evaluate  the  system  at  hand 
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•  vis  a  vis  the  mission  it  has  to  perform,  the  mission  requirements  must  be  expressed  in 
terms  of  the  MOPs  defined  for  the  system. 

Qualitatively,  the  mission  the  system  has  to  fulfill  is  to  deter  enemy  aircraft  from 
invading  friendly  territory,  without  killing  neutrals,  and  to  prevent  enemy  aircraft  from 
firing  missiles  aimed  at  friendly  assets.  In  term  of  the  MOPs  defined  in  chapter  5  and 
summarized  in  Table  6.1,  it  means  that  MOP1  must  be  greater  than  1,  that  MOP2  must  be 
as  close  as  possible  to  1,  and  that  MOP3  must  be  greater  than  1. 

Table  6.1:  MOPs  of  the  system 


MOP1 

compare  the  willingness  of  the  two  sides  to  keep  on  fighting. 
If  M0P1>1,  the  friends  are  winning  the  battle 

MOP2 

Ratio  of  the  neutrals  not  killed  by  friendly  forces  at  the 

end  of  the  battle  to  the  initial  number  of  neutrals 

MOP3 

Distance  of  the  enemy’s  aircraft  to  FSCL  at  the  end  of  the 
battle  divided  by  the  range  of  enemy's  missiles 

By  definition,  MOP2  is  less  than  1;  let  us  show  that  MOP1  is  less  than  l/(l-nf), 
and  that  MOP3  is  less  than  Rq/Rj. 

From  the  definition  of  MOP1,  one  can  deduce 

MOP1  =  ((l-n)/(l-nf))/((l-m)/(l-mf))  .  (6.1) 

Since  0  <  n  <  n^  and  0  <  m  <  mf  ,  it  follows  that 

MOP1  <  1/(1 -nf)  (6.2) 

If  MOP1  =  1/(1 -n^),  then  the  friendly  forces  are  winning  the  battle  without  loosing 
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anything. 

The  range  of  variation  of  MOP3  is  bounded  too;  indeed,  since 

MOP3  »  ((Rq  *  P2)  -  V  *  Tf)/Rj ,  it  follows  that 

MoresRo/Ri  (6.3) 


The  quantitative  requirements  are  assumed  to  be  as  follows: 

l/(l-nf)  z  MOP1  >  MOP10  =  1.1  (6.4) 

1  z  MOP2  z  MOP2q  *  0.8  (6.5) 

1  £  MOP3  Z  MOP30  *  1 .0  (6.6) 

Relation  6.4  requires  the  friends  to  win  the  battle  with  a  10%  margin;  inequality 
6.5  requires  that  no  more  than  20%  of  neutrals  be  killed,  and  relation  6.6  requires  the 
enemy  forces  to  be  stopped  before  they  have  crossed  the  line  from  which  friendly  positions 
are  within  range.  These  requirements  define  in  the  MOP  space  the  mission  locus  shown  in 
Fig.6.1.  One  should  note  that  this  mission  locus  is  bounded. 


Mission  locus 


Fig.6.1:  Projections  of  the  Mission  locus 


For  nf=0.6,  and  the  requirements  set  up  above,  if  Lm  denotes  the  mission  locus 


and  V(Lm)  its  volume,  one  can  compute 


V(Lm)  *  0.28 


(6.7) 


6.3  NOMINAL  IFFN  SYSTEM 
6.3.1  Introduction 

In  the  mathematical  model  that  has  been  presented  u  chapter  5,  five  parameters 
and  three  MOPs  have  been  defined:  we  are  interested  in  ploting  the  system  locus,  that  is  the 
locus  defined  by  the  variation  of  the  three  MOPs  as  the  five  parameters  are  varied  over  their 
admissible  ranges  independently  of  each  other. 

The  graphic  package  that  has  been  used  [Bohner  1986]  draws  a  line  between  two 
MOP  points  if  these  two  points  correspond  to  the  variation  of  one  parameter  at  a  time. 
Since  we  cannot  vary  the  five  parameters  on  a  three  dimensional  picture,  we  will  draw  a 
sequence  of  volumes  (partial  loci)  in  the  MOP  space:  for  each  of  these  volumes  two 
parameters  will  be  held  constant,  and  the  three  other  will  be  varied:  the  whole  system  locus 
will  be  defined  by  these  partial  loci  as  shown  in  Fig.6.2.  For  simplicity,  only  projections 
will  be  represented. 


System  Locus 


Fig.6.2:  System  locus  defined  by  four  partial  loci 
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6.3.2  System  Locus 


To  represent  the  system  locus,  as  mentioned  above,  we  will  consider  a  family  of 
partial  loci:  for  each  of  these  partial  loci,  parameters  Pj  and  P2  will  be  held  constant,  and 
parameters  P3,  P4,  P5  will  be  varied.  If  Pj  is  held  constant,  it  means  that  the  time  delay  to 
pass  information  from  one  node  of  the  system  to  another  is  kept  constant  Similarly,  if  P2 
is  held  constant  it  means  that  die  Air  Control  Procedure  is  not  changed. 

We  assume  the  ranges  of  variation  shown  in  Table.6.2  for  the  parameters: 


Table  6.2:  Parameters  ranges 


Definition 

Minimum 

Maximum 

p 

1 

Time  delay  to  pass  information 

0.10 

0.95 

P2 

Air  Control  Procedure 

0.75 

0.95 

P3 

Quality  of  identification 

0.75 

0.99 

P 

4 

Level  of  centralization 

0.50 

0.99 

P 

5 

Quality  of  Q&A  IFF  devices 

0.75 

0.99 

These  ranges  have  been  chosen  to  yield  realistic  and  physical  ranges  of  variation 
for  the  aggregate  quantities  defined  by  Logicon  [Logicon  1986](the  "Mops"  and  "Moes" 
considered  in  chapter  5). 

We  will  consider  four  partial  loci,  corresponding  to  the  maximal  and  minimal 
values  of  Pj  and  P2  : 

P1  =  P 1  max  a™1  p2  =  p2max  - >  Partial  Locus  #1 

pl  =  Plmin  ^d  p2  *  p2max  - >  Partial  Locus  #2 

P1  =plmin  and  P2  =  p2min  - >  Partial  Locus  #3 

P1  *  plmax  and  P2  =  P2min  - >  Pa™al  Locus  #4 


Before  showing  pictures  of  the  whole  locus,  let  us  set  P ^constant,  P2=constant, 
P3=constant  and  consider  the  set  of  MOP  points  obtained  by  varying  P4  and  P5:  this  will 
yield  a  "slice"  of  the  partial  loci  we  will  obtain  later,  and  give  insight  so  as  to  how  the  locus 
looks  like;  a  typical  "slice"  is  shown  in  Fig.6.3  and  Fig.6.4:  Fig.6.3  corresponds  to  the 
projection  of  this  slice  on  the  plane  (MOP1/MOP2),  and  Fig.6.4  on  the  projection  on  the 
plane  (MOP1/MOP3). 


The  purpose  of  these  two  figures  is  to  show  the  shape  of  a  typical  slice  of  the 
system  locus;  actual  and  accurate  plots  will  be  shown  later. 

One  can  note  an  irregularity  that  coresponds  to  MOPl=l,  that  is  to  the  change  in 
the  terminating  condition  (time  Tf):  if  MOP1  is  less  than  1  the  friends  give  up;  if  MOP1  is 
greater  than  1,  the  enemies  give  up. 

In  Fig.6.4,  MOP3  increases  if  MOP1  is  greater  than  1  and  if  MOP1  increases: 
indeed,  the  wider  the  margin  by  which  the  friendly  forces  are  winning,  the  farther  from  the 
FSCL  line  the  enemy  is  repulsed;  on  the  other  hand,  if  MOP1  is  less  than  1,  MOP3 
increases  as  MOP1  decreases:  indeed,  if  the  friends  are  loosing,  the  wider  the  margin  by 
which  they  are  loosing,  the  smaller  the  terminating  time  Tf  is;  the  smaller  the  terminating 
time,  the  smaller  the  distance  traveled  by  the  enemy  during  the  battle  is.  For  example,  if  the 
friends  give  up  immediately,  the  distance  traveled  by  the  enemy  during  the  battle  will  be 
equal  to  zero;  in  this  latter  case,  since  the  enemy  aircraft  are  not  repulsed,  they  will 
eventually  invade  the  friend's  territory. 

In  Fig.6.3,  a  vertical  (or  kinked)  line  is  drawn  for  each  value  of  P4,  that  is  for 
each  level  of  centralization  as  represented  by  P4  (P4=l  for  total  centralization);  the  greater 
the  value  of  P4,  the  farther  on  the  left  of  the  diagram  the  corresponding  vertical  line  is,  and 
the  smaller  MOP1  is.  It  means  that  the  lower  the  level  of  centralization,  the  greater  MOP1 
is,  that  is,  the  greater  the  chances  of  winning  the  battle  are;  this  is  the  result  of  a  trade-off 
between  the  accuracy  in  the  ID  process  and  the  time  needed  to  perform  the  identification: 
the  more  accurate  the  ID  is,  the  longer  it  takes.  It  turns  out  that  in  the  model,  the  time 
increase  in  the  ID  process  due  to  a  higher  level  of  centralization  is  the  most  important  of  the 
two  effects  (the  second  effect  being  an  increased  accuracy).  For  a  given  vertical  line  (that  is 
for  P4=constant),  the  greater  P5,  that  is  the  better  the  Question  and  Answer  IFF  devices 
are,  the  greater  MOP2  is:  it  means  that  the  better  the  Question  and  Answer  devices,  the 
greater  MOP2  is,  and  the  smaller  the  number  of  neutral  killed  is.  These  Q&A  devices  affect 
slightly  MOP1  except  around  MOPl=l  where  the  quality  of  these  devices  is  very 
important:  around  MOPl=l,  the  battle  can  be  won  or  lost  depending  on  the  quality  of  the 
Q&A  IFF  devices. 

For  the  next  plots,  P3  will  be  varied  with  P4  and  P5:  we  will  obtain  as  many  slices 
as  the  one  of  Figs. 6.3  and  6.4,  as  values  of  P3  considered.  One  should  recall  that  P3 
represents  the  quality  of  identification  (P3=l  for  perfect  ID  capabilities). 


Projection  on  the  plane  MOP1/MOP2 


In  Fig.6.5,  Fig. 6.6,  and  Fig. 6.7  projections  on  the  plane  (MOP1/MOP2)  are 
represented;  in  Fig.6.5  the  projection  of  partial  loci  #1  and  #4  (their  projections  on  the 
plane  MOP1/MOP2  are  the  same)  is  shown,  while  in  Fig.6.6  the  projection  of  partial  loci 
#2  and  #3  (their  projections  on  the  plane  MOP1/MOP2  are  the  same)  is  shown.  For  these 
two  latter  plots,  if  all  parameters  but  P3  are  fixed,  an  increase  in  P3  yields  a  higher  MOP1 
and  a  higher  MOP2:  the  better  the  ID  capabilities  of  the  system  the  easier  it  is  for  the 
friends  to  win,  and  the  smaller  the  number  of  neutrals  killed  by  the  system  is. 

Partial  loci  #1  and  #4  correspond  to  Pi=Pimax»  l^at  *s  l°ngest  time  delay  to 
pass  information  between  two  nodes  of  the  system;  on  the  other  hand,  partial  loci  #2  and 
#3  correspond  to  the  shortest  time  delay  to  pass  information  between  two  nodes.From 
these  two  loci  one  can  check  the  consistency  of  the  model:  the  shorter  the  time  to  exchange 
information  between  nodes,  the  greater  MOP1,  and  the  greater  the  chances  of  winning  the 
batde. 

In  Fig.6.7  the  projection  of  the  entire  system  locus  on  the  plane  MOP  1/MO P2  is 
shown  ;  it  is  obtained  by  superposing  the  two  previous  plots. 

M0P2 
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Fig.6.5:  Partial  Loci  #1  and  #4  projected  on  Plane  MOP1/MOP2 
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Fig.6.6:  Partial  Loci  #2  and  #3  projected  on  Plane  MOP1/MOP2 


Fig.6.7:  Entire  System  Locus  projected  on  Plane  MOP1/MOP2 


In  Fig.6.8  the  projections  of  partial  loci  #1  and  #4  on  the  plane  MOP1/MOP3  are 
shown  .  The  upper  pan  corresponds  to  partial  locus  #1,  or  to  the  highest  quality  of  Air 
Control  Procedure  (P2  -  P2max)-  l°wer  pan  of  Fig.6.8  corresponds  to  partial  locus 
#4  and  to  a  low  quality  Air  Control  Procedure  (ACP);  the  better  the  ACP  ,  the  greater 
MOP3:  indeed,  with  a  good  ACP  one  can  detect  an  enemy  aircraft  early  and  therefore  stop 
it  far  away  from  the  FSCL.  The  angle  at  MOPl=l  corresponds  to  the  change  in  the 
terminating  conditions;  it  corresponds  to  the  irregularity  already  noted  on  previous  plots 
around  MOP1  =  L 

In  Fig.6.9  the  projections  of  partial  loci  #2  and  #3  on  the  plane  MOP1/MOP3  are 
shown  .  The  upper  part  corresponds  to  partial  locus  #  2,  and  the  lower  part  corresponds  to 
partial  locus  #3;  as  in  Fig.6.8,  the  better  the  ACP  the  greater  MOP3  is. 

In  Fig.6.10  the  projection  of  the  entire  locus  defined  as  in  Fig.6.3  is  shown  . 


M0P3 


Fig.6.8:  Partial  Loci  #1  and  #4  projected  on  Plane  MOP1/MOP3 


6.3.3  Measures  of  Effectiveness 

If  Ls  designates  the  system  locus  and  designates  the  mission  locus,  and  if 
V(L)  is  the  volume  of  L,  then,  to  compute  the  effectiveness  of  the  mathematical  model  of 
the  system,  one  must  evaluate  V(LS  «  L^),  and  VCL^)  or  VfL^  depending  on  the  MOE 
one  is  interested  in.  With  the  definitions  of  chapter  2,  Ej  will  measure  how  well  the  system 
capabilities  are  used,  E2  will  measure  how  well  the  mission  is  covered  by  the  system,  and 
E3  will  measure  the  degree  of  mismatch  between  system  and  mission. 

For  the  basic  operating  point  considered  in  this  section,  we  have 


V(LS  o  Ljjj)  «  0.020  (6.12) 

V(LS)«  0.068  (6.13) 

VfLjn)  =  0.28  (6.6) 

Therefore 

E!  -  0.292  (6.14) 

E2- 0.071  (6.15) 

E3  »  0.061  (6. 16) 


MOEs  E2  and  E3  are  very  small  because  of  the  size  of  the  mission  locus  which 
takes  into  account  such  unrealistic  events  as  the  possibility  for  the  friends  of  winning  the 
battle  without  loosing  any  asset  Therefore,  the  degree  of  coverage  of  the  mission  locus  by 
the  system  locus  (E2)  is  very  low  and  the  degree  of  mismatch  (I-E3)  between  the  two  loci 
is  very  high. 

6.3.4  Conclusion 

In  this  section  the  results  of  the  System  Effectiveness  Analysis  applied  to  the 
nominal  system  have  been  presented.  Since  the  loci  corresponding  to  other  basic  operating 
points  are  qualitatively  the  same  as  the  ones  shown  above,  in  tie  next  section  of  this 
chapter,  only  the  numerical  results  of  the  SEA  methodology  will  be  presented. 


6.4  NUMERICAL  RESULTS  FOR  THE  REMAINING  BASIC  OPERATING  POINTS 


6.4. 1  Removal  of  the  NE-3  A 

For  this  basic  operating  point  defined  in  chapter  5  we  have  the  following  results 


El  -  0.059 

(6.17) 

e2  -  0.01 1 

(6.18) 

E3  -  0.009 

(6.19) 

Therefore,  the  removal  of  the  NE-3  A  yields  very  significant  reductions  in  the 
MOEs:  the  resulting  configuration  has  an  extremely  poor  effectiveness  whatever  the  MOE 
considered  is. 

6.4.2  Removal  of  the  JTIDS 

For  this  basic  operating  point  defined  in  chapter  5  we  have  the  following  results 


Et  -0.113 

(6.20) 

E2  -  0.012 

(6.21) 

e3  -  0.01 1 

(6.22) 

To  the  removal  of  the  JTIDS  corresponds  smaller  MOEs  than  in  section  6.3.3,  but 
the  reductions  are  not  as  large  as  with  the  removal  of  the  NE-3A;  therefore,  the  NE-3A 
appears  to  be  more  important  to  the  system  than  the  JTIDS. 


6.4.3  Removal  of  the  NE-3A  and  of  the  JTIDS 

For  this  basic  operating  point  defined  in  chapter  5,  the  intersection  of  the  system 
locus  and  the  mission  locus  is  the  empty  set  (MOP3  is  always  less  than  1 );  thus  we  have 
the  following  results 


6.4.4  Removal  of  the  JTIDS  and  inclusion  of  an  additional  NE-3A 


For  this  basic  operating  point  defined  in  Chapter  5  we  have  the  following  results 


Ei  -0.167 

(6.26) 

E2- 0.021 

(6.27) 

E3  -  0.019 

(6.28) 

6.4.5  Summary  of  results 

The  results  obtained  in  this  chapter  are  summarized  in  Table  6.3. 


Table  6.3:  Summary  of  results 


Sysrem^^,,*«^_  MOEs 
configuration 

E 

1 

E2 

E3 

Nominal 

0.292 

0.071 

0.061 

Without 

NE-3A 

0.059 

0.011 

0.009 

Without 

JTIDS 

0.113 

0.012 

0.011 

Without 

NE-3A  and  JTIDS 

0 

0 

0 

Additional  NE-3A 

Without  JTIDS 

0.167 

0.021 

0.019 

6.5  CONCLUSION  ' 

In  this  chapter,  the  System  Effectiveness  Analysis  methodology  has  been  applied 
to  the  mathematical  model  describing  the  IFFV  system.  The  resulting  measures  are  MOEs 
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for  the  simplified  model 

In  the  next  chapter,  the  real  system  will  be  evaluated  by  using  the  algorithm 
described  in  Chapter  3. 


CHAPTER  7 

APPLICATION:  IMPROVEMENT  OF  AN  ACTUAL  SYSTEM 


7.1  INTRODUCTION 

In  this  chapter  the  methodology  developed  in  chapter  3  and  chapter  4  is  applied  to 
the  IFFN  system  presented  in  chapter  5. 

Five  basic  operating  points  have  been  defined  for  the  IFFN  system;  we  assume 
that  the  system  at  hand  is  the  one  corresponding  to  the  fourth  basic  operating  point:  the 
Joint  Tactical  Information  Distribution  System  (JTIDS)  and  the  NATO  airborne  early 
warning  (NE-3A)  are  not  included  in  the  initial  system,  and  one  is  interested  in  determining 
the  optimal  sequence  of  modification  for  the  system.  These  modifications  can  be:  adding 
the  JTIDS  to  the  system,  or  adding  the  NE-3A.  The  optimal  sequence  of  improvement  will 
be  determined  for  the  actual  system  as  opposed  to  the  model  presented  in  chapter  5:  for 
each  stage  of  the  process,  the  methodology  presented  in  chapter  3  will  be  used. 

In  the  first  part  of  this  chapter,  the  effectiveness  of  the  actual  system  at  each  basic 
operating  point  will  be  evaluated.  The  second  part  will  deal  with  the  improvement 
algorithm. 


7.2  ACTUAL  SYSTEM  EFFECTIVENESS 
7.2.1  Introduction 

The  results  of  the  system  effectiveness  analysis  will  be  presented  extensively  for 
the  final  system,  that  is  the  one  obtained  after  adding  the  JTIDS  and  the  NE-3A  to  the 
initial  system.  The  reason  for  this  choice  is  that  one  is  interested  mostly  in  the  final  system 
as  opposed  to  any  intermediate  stage.  For  the  four  other  basic  operating  points  only 
numerical  results  will  be  presented. 


7.2.2  Final  system:  actual  system  locus 

The  presentation  of  the  results  in  this  section  follows  the  same  format  as  in  chapter 
6  where  the  system  locus  of  the  mathematical  model  has  been  introduced.  To  determine  the 
actual  system  locus,  we  will  go  through  the  flowchart  of  Fig  3.9. 

Step  1:  Determination  of  the  model  locus. 

This  step  has  been  completed  in  chapter  6  where  the  results  of  the  SEA 
methodology  applied  to  the  mathematical  model  of  the  IFFN  system  have  been  presented. 
The  locus  is  shown  in  Figs.  6.5  to  6.13  in  Chapter  6. 

Step  2:  Selecdon  of  points  on  the  model  locus. 

As  mentioned  in  chapter  3,  we  inscribe  the  model  locus  in  a  parallilepiped,  and 
choose  the  points  of  contact  between  the  model  locus  and  the  parallilepiped  (or  the  center 
of  gravity  of  these  points).  In  what  follows  a  row  vector  &  is  constructed  with  elements  of 
three  MOPs: 

5  =  [  MOP1  ,  MOP2  ,  MOP3  ]  (7.1) 

For  the  locus  obtained  in  chapter  6  and  shown  in  Figs. 6.5  to  6.13,  six  points  of  contact  are 
obtained: 


Xdl  =  [  1.639 ,0.999  ,  1.203] 

(7.2) 

£&  =  [  1.432,0.999,  1.081] 

(7.3) 

2^3  =  [  1.639 ,0.999 , 1.403] 

(7.4) 

2^4  =  [  0.911  ,0.687,0.867] 

(7.5) 

2^5  =  [  1.006 ,0.611 ,0.826] 

(7.6) 

2^6  =  [  1.028 ,0.987 ,0.610] 

(7.7) 

The  six  vectors  obtained  above  represent  the  entire  system  locus  as  opposed  to  any 
of  its  region.  The  three  first  vectors  correspond  to  maximum  values  for  the  MOPs  in  the 
system  locus:  x^j  corresponds  to  the  maximum  value  of  MOP1,  x^-?  corresponds  to  the 
maximum  value  of  MOP2,  and  X31  corresponds  to  the  maximum  value  of  MOP3.  These 
three  first  vectors  are  in  the  mission  locus.  One  should  note  in  Fig. 6. 7  that  the  number  of 
contact  points  is  infinite  for  MOP2=0.999:  x<p  is  defined  as  the  center  of  gravity  of  this 
infinite  number  of  points.  Similarly  in  Fig. 6. 10,  one  can  remark  that  the  number  of  points 
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corresponding  to  the  maximum  value  of  MOP1  is  infinite:  x^i  is  thus  defined  as  the  center 
of  gravity  of  these  points.  The  numerical  values  of  the  MOPs  for  can  be  interpreted  as 
follows:  MOP  1=1. 639  means  that  the  friends  are  winning  the  battle  with  a  margin  of  64% 
or  so:  this  is  the  largest  possible  margin  for  the  system.  MOP2=0.999  means  that  less  than 
1%  of  the  neutrals  are  killed  by  the  friendly  forces.  Lastly  MOP3=  1.203  means  that  the 
enemy  forces  are  stopped  before  they  have  been  able  to  fire  any  missiles. 

The  three  last  vectors  correspond  to  minimum  values  for  the  MOPs  in  the  system 
locus:  2^4  corresponds  to  the  minimum  value  of  MOP1, 2^5  corresponds  to  the  minimum 
value  of  MOP2,  and  2^6  corresponds  to  the  minimum  value  of  MOP3.  These  three  last 
vectors  are  not  in  the  mission  locus.  One  can  note  in  Fig.6.10  that  an  infinite  number  of 
points  correspond  to  the  minimum  value  of  MOP1:  therefore,  2^4  is  defined  as  the  center 
of  gravity  of  these  points.  For  this  vector  2^4  ,  the  friends  loose  the  battle  (M0P1<1),  kill 
more  than  30%  of  the  neutrals  and  fail  to  prevent  the  enemy  from  firing  missiles 
(M0P3<1). 

Step  3:  Inversion  algorithm. 

For  each  of  the  points  2&di  determined  above,  we  compute  a  parameter  value  pj 
such  that  2^=  f(pj),  where  f  denotes  the  mathematical  model  of  the  system.  If  one  notes  a 
parameter  vector  p  as  a  row  vector 


p  =  (pI  ,P2,p3,P4,P5]  (7.8) 

then,  the  parameter  vectors  corresponding  to  the  x^j  determined  above  are 

Pj  =  [  0.205 , 0.850 , 0.990 , 0.500 , 0.904  ]  (7.9) 

P2  =  [  0.554 , 0.850 , 0.990 , 0.694 , 0.911  ]  (7.10) 

p3  =  [  0.204 , 0.950 , 0.990 , 0.500 , 0.900  ]  (7.11) 

C4  =  [  0.685 , 0.850 , 0.750 , 0.846 , 0.933  ]  (7. 12) 

fi5  =  [  0.521  ,  0.851  ,  0.821  ,  0.652 , 0.750  ]  (7.13) 

P6  =  [  0.873 , 0.750 , 0.874 , 0.990 , 0.990  ]  (7. 14) 


These  values  were  obtained  using  the  algorithm  described  in  Section  3.3  of 
Chapter  3.  As  expected,  they  are  within  the  admissible  range  of  variation  in  the  parameter 
space  as  defined  by  Table  6.2. 

The  first  parameter  vector  pj  corresponds  to  or  to  the  maximum  value  for 
MOP1.  This  maximum  value  of  MOP1  corresponds  to  the  maximum  margin  by  which  the 


battle  is  won.  It  is  obtained  for  a  small  but  not  minimum  time  delay  to  pass  information 
(first  parameter),  a  medium  quality  of  Air  Control  Procedure  (second  parameter),  a 
maximum  quality  of  identification  (third  parameter),  a  minimum  level  of  centralization 
(fourth  parameter),  and  a  good  (but  not  maximum)  quality  of  Q&A  IFF  devices.  Table  7.2 
summarizes  the  physical  significance  of  the  parameter  vectors  obtained  using  the  inversion 
algorithm. 


Table  7.1:  Physical  significance  of  the  parameter  vectors 


Time  delay 
to  pass 
Information 

Quality  of 
Air  Control 
Procedure 

Quality  of 

Identification 

Level  of 

Centralization 

Qualitv  of 
Q&A  IFF 
Devices 

igggwjjl 

"i 

Small 

Medium 

Maximum 

Minimum 

High 

E2 

Medium 

Medium 

Maximum 

Medium 

High 

E3 

Small 

Maximum 

Maximum 

Minimum 

High 

E  4 

High 

Medium 

Minimum 

High 

High 

Minimum 

MOP2 

E  5 

Medium 

Medium 

Medium 

Medium 

Minimum 

Minimum 

MOP3 

E  6 

High 

Minimum 

Medium 

Maximum 

Maximum 

MOP1  and  the  margin  by  which  the  battle  is  won  is  strongly  linked  to  the  quality 
of  identification  (DD):  the  maximum  MOP1  is  obtained  for  the  maximum  quality  of  ID  and 
the  minimum  level  of  centralization,  and  the  minimum  MOP1  is  obtained  fo~  the  minimum 
quality  of  ID  and  the  highest  level  of  centralization.  The  fact  that  an  increase  in  the  quality 
of  ID  improves  MOP1  is  easy  to  predict.  The  greater  the  level  of  centralization,  the  lower 
MOP1  is:  this  is  the  result  of  a  trade-off  outlined  in  Chapter  6  betv/een  the  increase  in  the 
accuracy  of  the  ID  process  due  to  a  higher  level  of  centralization,  and  the  increase  in  the 
time  needed  to  perform  this  ID  also  due  to  a  higher  level  of  centralization:  i:  turns  out  that 
the  second  of  the  two  effects  is  the  most  important  one,  thus  reducing  MOP1.  One  should 
also  note  that  MOP1  depends  on  the  time  delay  to  pass  information  between  nodes:  the 
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smaller  this  time  delay,  the  faster  the  response  of  the  system,  and  the  greater  MOP1  is. 

MOP2  appears  to  be  linked  to  the  quality  of  ID  and  to  the  quality  of  the  Q&A 
IFFN  devices  which  provide  local  ID  information:  the  greater  the  quality  of  ID  and  the 
better  the  local  ID  information,  the  lower  the  number  of  neutrals  killed  by  the  friendly 
forces  is. 

MOP3  depends  mostly  on  the  Air  Control  Procedure  (ACP),  and  on  the  time  delay 
to  pass  information  betweer  nodes:  the  better  the  ACP  and  the  smaller  the  time  delay  to 
pass  information,  the  gr  „cr  MOP3  is.  Indeed  with  a  good  ACP,  one  is  able  to  detect  the 
enemy  far  in  the  detection  volume,  and  the  smaller  the  time  delay  to  pass  information,  the 
faster  the  response  of  the  system  and  the  greater  MOP3  is. 

Step  4:  Experimental  results. 

At  this  stage,  experiments  are  run  at  the  parameter  vectors  determined  at  stage  3. 
Since  we  cannot  run  experiment  on  the  actual  system  for  the  purpose  of  this  thesis,  a 
mathematical  model  which  is  slightly  different  from  the  one  introduced  in  chapter  6  has 
been  used.  Let  us  introduce  briefly  this  modified  model  that  "represents"  the  actual  system; 
this  model  is  the  same  as  the  one  of  chapter  6  except  that  each  time  a  parameter  vector 

C=[P1,P2,P3,P4,P5] 

is  used  in  an  equation,  this  parameter  p  is  replaced  by 

ft*  =  [  Fj  ,  P-2  .  P'3  •  P'4  •  P’5  1 

with,  P'i  =  ^  *  (1  +  (Pi-Pi0)*Hp  -  Poll )  (7. 15) 

Therefore,  the  parameter  values  in  the  modified  model  are  affected  by  the  distance  of  the 
parameter  vector  considered  to  the  parameter  vector  pq  introduced  in  chapter  3;  for  the  case 
at  hand  we  choose 

PO  =  [0.6 , 0.85 , 0.87 , 0.75 , 0.87  ]  (7.16) 

This  vector  has  been  selected  because  it  is  a  central  point  in  the  parameter  locus;  the 
simplified  model  is  calibrated  at  this  point  since  for  p  =  pq  ,  and  the  model  and  the  actual 
system  (as  represented  by  the  modified  model)  coincide. 

The  pseudo-experimental  values-obtained  by  exercising  the  modified  model  are 
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2^*1  1.513,0.999, 1.128] 
^2  =  t  1-^36 , 0.999 , 1.082  ] 
2^3  =  [  1.509 , 0.999 , 1.372  ] 
2^4  =  [0.911 ,0.688,0.868  ] 
2^5  =  [  0.997 , 0.617 , 0.830  ] 
2^6  =  [  1.006 , 0.903 , 0.565  ] 


(7.17) 
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Step  6:  Transformation  from  the  model  locus  into  the  actual  locus. 

If  xm  is  a  point  in  the  model  locus  and  if  xa  is  the  corresponding  point  in  the  actual 
locus,  the  least  square  procedure  presented  in  chapter  3  yields  the  following  transformation 


%  “  ^(Xm)  -  Lxm  +  Y , 


(7.23) 


where  L  is  a  linear  transformation  (defined  by  a  3x3  matrix)  and  V  a  constant 
translation  vector.  For  the  example  at  hand  we  have 
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-0.151 


L  =  0.042 


0.813 


0.0008 
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1.127 


0.142 


V=  0.017 


0.043 


(7.24) 


Step  7:  Construction  of  the  actual  locus. 

For  each  point  in  the  parameter  locus  we  apply  A  =  Tof  where  "o"  denotes  the 
composition  of  two  functions  and  where  "F ’  stands  for  the  mathematical  function  that  maps 
the  parameter  locus  into  the  model  system  locus. 

Since  we  have  considered  the  actual  system  to  be  "represented"  by  a  modified 
model  which  has  the  same  structure  as  the  simplified  one,  the  actual  locus  and  the  model 
locus  have  the  same  shape  and  the  same  structure.  If  real  experiments  could  be  run,  the 
actual  locus  might  be  very  different  from  the  model  locus. 

For  the  example  at  hand,  since  the  shape  of  the  actual  locus  in  qualitatively  the 
same  as  the  one  of  the  model  locus,  only  comparisons  of  the  two  loci  will  be  presented:  in 
the  following  plots,  the  contours  of  the  projections  of  both  the  model  and  the  actual  locus 
are  shown;  these  projections  are  done  on  the  planes  MOP1/MOP2  (Fig.7.1),  MOP1/MOP3 
(Fig. 7. 2),  MOP2/MOP3  (Fig.7.3). 
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Fig.7.3:  Actual  and  Model  Loci  projected  on  Plane  MOP3/MOP2 


7.2.3  Effectiveness  of  the  actual  system 


From  the  actual  locus  of  the  nominal  system  constructed  in  section  7.2.2,  one  can 
evaluate  the  effectiveness  of  the  system.  If  Lsa  denotes  the  actual  system  locus,  then  with 
the  notation  of  Chapter  2,  the  measures  of  effectiveness  for  the  actual  system  are: 


Ei  =  V(LsanLm)/V(Lsa) 


(7.25) 


Ez  -  V(L$a  n  LjjjVV^Ljh) 


(7.26) 


E3  =  V(LsanLm)/V(LsauLIT1) 


(7.27) 


For  the  nominal  system  we  have 


Ex  =  0.200 


E2  =  0.162 


Et  =  0.152 


Only  numerical  results  are  presented  for  the  remaining  four  basic  operating  points. 
In  Table  7.2  the  results  of  the  effectiveness  analysis  are  presented  for  the  five 
configurations  of  the  actual  system. 
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Table  7.2:  Effectiveness  of  the  actual  system 


MOEs 

Actual  — . 

System 

configuration 

E, 

E  2 

E 

3 

Nominal 

0.300 

0.049 

0.044 

Without 

NE-3A 

0.075 

0.006 

0.005 

Without 

JTTDS 

0.172 

0.006 

0.006 

Without 

NE-3A  and  JTIDS 

0.022 

0.002 

0.002 

Additional  NE-3A 

Without  JTIDS 

0.200 

0.162 

0.152 

In  order  to  enable  one  to  compare  the  results  obtained  for  the  model  and  the  ones 
obtained  for  the  actual  system  Table  6.3  of  Chapter  6  is  reproduced  in  Table  7.3. 

Table  7.3:  Effectiveness  based  on  the  model 


System^>^^  MOEs 
configuration 

E 

1 

E2 

E 

3 

Nominal 

0.292 

0.071 

0.061 

Without 

NE-3A 

0.059 

0.011 

0.009 

Without 

JTIDS 

0.113 

0.012 

0.011 

Without 

NE-3A  and  JTIDS 

0 

0 

0 

Additional  NE-3A 

Without  JTIDS 

0.167 

0.021 

0.019 

The  values  of  Ej  obtained  for  the  actual  system  (Table  7.2)  are  slightly  greater 
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than  the  ones  obtained  for  the  model  (Table  7.3);  in  particular,  the  effectiveness  E|  of  the 
actual  system  without  NE-3A  and  JTIDS  is  very  small,  but  not  equal  to  zero.  It  means  that 
the  capabilities  of  the  actual  system  are  better  used  than  one  could  have  thought  by 
studying  the  model  only.  On  the  other  hand,  for  the  first  three  basic  operating  points,  the 
values  of  E2  and  E3  are  slightly  smaller  for  the  actual  system  than  they  are  for  the  model: 
the  degree  of  coverage  of  the  mission  is  smaller  for  the  actual  system  than  for  the  model, 
and  the  degree  of  misfit  between  mission  and  system  loci  is  greater  for  the  actual  system 
than  for  the  model.  For  the  last  two  basic  operating  points,  the  degree  of  coverage  of  the 
mission  is  smaller  for  the  actual  system  than  for  the  model,  and  the  degree  of  misfit 
between  mission  and  system  loci  is  greater  for  the  actual  system  than  for  the  model. 

7.3  SYSTEM  PLANNING 
7.3.1  Assumptions 

In  this  section,  the  dynamic  programming  algorithm  introduced  in  Chapter  4  is 
applied  to  the  "actual  system".  We  assume  that  the  initial  system  is  the  one  without  NATO 
Airborne  Early  Warning  (NE-3A)  and  Joint  Tactical  Information  Distribution  System 
(JTIDS)  and  we  consider  a  two  stage  evolution:  the  initial  system  is  denoted  by  Xq  ;  at 
stage  1,  the  system  is  Xj,  and  at  stage  2  it  is  X2.  At  each  stage,  one  can  add  either  the 
JTIDS  or  the  NE-3A  but  not  both.  Therefore  the  set  of  possible  decisions  is  assumed  to  be 
known.  Another  assumption  is  that  the  only  MOE  of  interest  for  the  developer  is  H  j :  it 
means  that  the  developer  is  interested  in  knowing  how  well  the  capabilities  of  the  system 
are  used.  The  goal  of  the  evolution  process  is  the  maximization  of 

V  =  E1(1)  +  E1(2)  (7.28) 

where  E|(i)  stands  for  Ej  at  stage  i. 

With  the  set  of  possible  decisions  considered  in  this  section,  it  is  possible  for  the 
final  system  to  be  composed  of  the  initial  system  Xq  to  which  have  been  added  two  JTIDS: 
this  doesn't  make  any  sense,  and  this  possibility  should  be  eliminated.  The  branch  leading 
to  this  particular  state  will  not  be  considered  in  the  decision  tree  to  be  sketched  as  in 
Chapter  4. 


7.3.2  Application 


With  these  assumptions  and  the  framework  provided  in  Chapter  4,  one  can  draw  a 
decision  tree  similar  to  the  one  of  Fig.4.3.  In  Fig.7.4  this  decision  tree  is  shown:  at  each 
stage,  the  upper  branch  corresponds  to  the  addition  of  a  NE-3A,  and  the  lower  branch 
corresponds  to  the  addition  of  a  JTIDS. 
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Fig.7.4:  Optimal  evolution  sequence  for  the  IFFN  system 


From  Fig.7.4,  one  can  deduce  the  optimal  sequence  of  improvements  for  the 
initial  system  Xq:  first  add  to  the  initial  system  the  NE-3A,  to  yield  Xj;  second  add  the 
JTIDS  to  Xj  to  yield  the  final  system.  The  optimal  value  of  V  defined  by  equation  7.28  is 
then 


The  dynamic  programming  algorithm  introduced  in  Chapter  4  has  been  used  for 
the  actual  IFFN  system  whose  effectiveness  has  been  analysed  in  section  7.2.  It  allows 
one  to  plan  the  evolution  of  the  system:  in  the  IFFN  example,  the  optimal  sequence  of 
modification  is.  First,  add  an  early  warning  component  (NE-3A)  whose  function  is  to 
increase  the  detection  volume  of  the  system  and  to  increase  the  probabilities  of  detecting 
and  identifying  an  aircraft.  Second,  add  a  high  capacity  information  distribution  system 
(JTTDS)  whose  function  is  to  provide  faster  communication  between  the  nodes  of  the 
system,  and  to  increase  the  probabilities  of  detecting  and  identifying  an  aircraft  because  of 
higher  communication  reliability.  Therefore,  for  the  initial  system,  increasing  the  detection 
volume  is  the  first  priority.  The  second  priority  is  the  addition  of  a  component  providing 
the  system  with  fast  communication  capabilities. 

7.4  CONCLUSION 

In  this  chapter,  the  methodology  developed  in  Chapter  3  and  Chapter  4  has  been 
applied  to  the  IFFN  system  presented  in  Chapter  5:  the  procedure  to  evaluate  the 
effectiveness  of  an  actual  system  has  been  demonstrated  as  well  as  the  system  planning 
procedure. 

The  methodology  applied  in  this  chapter  provides  the  system  developer  with  a 
powerful  tool  since  it  enables  him  to  make  the  best  possible  use  of  a  test  bed:  it  allows  him 
to  assess  the  effectiveness  of  different  configurations  of  the  system  by  running  the  smallest 
possible  number  of  experiment;  it  also  allows  him  to  determine  an  optimal  sequence  of 
improvement  as  the  system  goes  from  one  configuration  to  another. 

The  next  chapter  concludes  this  thesis  and  gives  directions  for  further  research. 
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CHAPTER 8 

CONCLUSIONS  AND  DIRECTIONS  FOR 
FURTHER  RESEARCH 


8.1  CONCLUSIONS 

As  pointed  out  in  Chapter  1,  since  a  large  scale  system  often  cannot  be  exercised 
to  obtain  needed  performance  data,  one  must  build  a  test  bed.  Such  a  test  bed  provides  a 
means  to  gather  data  on  the  system;  it  also  provides  a  means  for  simulating  different 
configurations  of  the  system.  In  this  thesis,  a  methodology  to  make  the  best  possible  use 
of  a  test  bed  has  been  presented. 

The  first  step  in  the  methodology  aims  at  determining  the  smallest  number  of 
experiments  that  need  to  be  run  to  evaluate  the  effectiveness  of  an  actual  system.  A  basic 
assumption  is  that  a  simplified  mathematical  model  of  the  system  at  hand  can  be  obtained. 
This  model  is  used  to  determine  experiments;  then  it  is  used  together  with  experimental 
results  to  yield  the  effectiveness  of  the  system.  This  first  step  provides  the  system 
developer  with  a  powerful  tool  since  it  enables  him  to  select  and  design  a  very  small 
number  of  experiments  to  run  on  the  test  bed  in  order  to  evaluate  its  effectiveness. 

In  the  second  step,  a  system  planning  procedure  for  optimizing  the  evolution  of  a 
large  scale  system  is  used.  After  evaluating  the  effectiveness  of  different  configurations  of 
the  system  with  the  first  step,  this  procedure  enables  the  system  developer  to  select  the  best 
evolution  path  for  the  system  at  hand.  The  basic  assumption  underlying  this  procedure  is 
that  the  possible  improvements  are  constrained  to  belong  to  a  fixed  set  of  improvements. 

This  methodology  has  been  illustrated  by  applying  it  to  an  air  defense  system 
currently  under  development,  and  known  as  the  "Identification  Friend  Foe  Neutral 
system",  or  "IFFN  system".  Four  different  configurations  of  this  system  have  been 
considered,  and  the  effectiveness  of  the  system  in  each  of  these  configurations  has  been 
evaluated.  Then,  the  system  planning  procedure  has  been  applied  to  these  four 
configurations  on  the  basis  of  the  effectiveness  evaluation,  to  determine  an  optimal 
sequence  of  modifications  for  the  system. 
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8.2  DIRECTIONS  FOR  FURTHER  RESEARCH 


J 


Continuation  of  this  research  should  proceed  as  the  methodology  gets  applied  to 
new  examples.  In  this  section,  recommendations  for  future  research  are  given;  they  focus 
on  the  definition  of  boundaries  for  a  system,  and  on  the  optimization  of  a  system's 
evolution. 

Physical  and  temporal  boundaries  of  a  system:  The  definitions  and  guidelines 
provided  in  this  thesis  to  draw  the  boundaries  of  a  system  should  be  further  tested  on 
actual  examples  and  expanded,  if  necessary.  In  particular,  the  notions  of  evolving  system 
and  of  temporal  boundary  for  a  system  should  be  further  developed. 

Optimal  evolution  of  a  system:  Further  research  to  expand  the  framework  of  the 
dynamic  programming  algorithm  presented  in  this  thesis  will  be  helpful.  Indeed,  a  basic 
assumption  underlying  the  algorithm  is  that  the  set  of  decisions  that  can  be  taken  by  the 
system  developer  is  known  at  the  planning  stage  of  the  development.  This  assumption  is 
justified  for  systems  that  cannot  be  built  at  once  because  of  time  and  budget  constraints.  On 
the  other  hand,  if  a  system  is  to  integrate  at  each  stage  of  its  evolution  the  technological 
state  of  the  art,  one  cannot  assume  that  the  set  of  all  possible  improvements  will  be  known 
at  the  planning  stage.  For  such  systems,  relaxing  this  particular  assumption  is  crucial,  if 
one  wants  to  optimize  their  evolution. 
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